2020-11-20T13:45:50.777 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-20T13:45:50.779 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi --owner scheduled_teuthology@teuthology --block --archive /home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628501 --description fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_latest} tasks/auto-repair} --lock -- /tmp/teuthology-worker.2rzi_t2k.tmp 2020-11-20T13:45:50.828 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T13:45:50.893 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628501 branch: octopus description: fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_latest} tasks/auto-repair} email: ceph-qa@ceph.io first_in_suite: false job_id: '5628501' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: centos os_version: '8.1' overrides: admin_socket: branch: octopus ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - force file system read-only - bad backtrace - MDS in read-only mode - \(MDS_READ_ONLY\) sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 owner: scheduled_teuthology@teuthology priority: 1000 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.1 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - client.2 - client.3 sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/git.ceph.com_ceph_octopus/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_auto_repair teuthology_branch: master timestamp: 2020-11-16_04:10:02 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.6029 2020-11-20T13:45:50.893 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_octopus/qa; will attempt to use it 2020-11-20T13:45:50.894 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_octopus/qa/tasks 2020-11-20T13:45:50.894 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-20T13:45:50.896 INFO:teuthology.task.internal:Checking packages... 2020-11-20T13:45:50.914 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'a7b62340ad27b26642bb54ec916a97c7e1f07a34' 2020-11-20T13:45:50.915 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-20T13:45:50.915 INFO:teuthology.packaging:ref: None 2020-11-20T13:45:50.916 INFO:teuthology.packaging:tag: None 2020-11-20T13:45:50.916 INFO:teuthology.packaging:branch: octopus 2020-11-20T13:45:50.916 INFO:teuthology.packaging:sha1: a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:45:50.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2020-11-20T13:45:51.193 INFO:teuthology.task.internal:Found packages for ceph version 15.2.6-589.g08106742 2020-11-20T13:45:51.193 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-20T13:45:51.204 INFO:teuthology.task.internal:no buildpackages task found 2020-11-20T13:45:51.204 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-20T13:45:51.221 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-20T13:45:51.221 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T13:45:51.279 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628501'} 2020-11-20T13:45:51.298 DEBUG:teuthology.lock.ops:locked ubuntu@smithi117.front.sepia.ceph.com, ubuntu@smithi085.front.sepia.ceph.com 2020-11-20T13:45:51.355 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3590692268>, signals=[15]) 2020-11-20T13:45:51.951 INFO:teuthology.provision.fog.smithi117:Scheduling deploy of centos 8.1 2020-11-20T13:45:52.027 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of centos 8.1 2020-11-20T13:45:55.475 INFO:teuthology.orchestra.console:Power off smithi117 2020-11-20T13:45:55.476 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T13:45:55.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:45:55.613 INFO:teuthology.orchestra.console:Power off for smithi117 completed 2020-11-20T13:45:55.714 INFO:teuthology.orchestra.console:Power on smithi117 2020-11-20T13:45:55.714 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T13:45:55.733 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:45:55.790 INFO:teuthology.orchestra.console:Power off smithi085 2020-11-20T13:45:55.791 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi085.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T13:45:55.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi085.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:45:55.928 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2020-11-20T13:45:56.029 INFO:teuthology.orchestra.console:Power on smithi085 2020-11-20T13:45:56.029 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi085.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T13:45:56.048 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi085.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:45:59.752 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:00.069 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi085.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:03.865 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:04.074 INFO:teuthology.orchestra.console:Power on for smithi117 completed 2020-11-20T13:46:04.176 INFO:teuthology.provision.fog.smithi117:Waiting for deploy to finish 2020-11-20T13:46:04.179 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi085.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T13:46:04.388 INFO:teuthology.orchestra.console:Power on for smithi085 completed 2020-11-20T13:46:04.489 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish 2020-11-20T13:48:39.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:48:48.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:48:54.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:48:57.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:03.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:06.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:12.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:15.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:21.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:24.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:30.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:33.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:39.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:49:48.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:05.307 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T13:50:05.308 INFO:teuthology.orchestra.run.smithi085:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T13:50:13.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:22.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:22.502 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T13:50:22.503 INFO:teuthology.orchestra.run.smithi117:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T13:50:26.446 INFO:teuthology.provision.fog.smithi085:Node is ready 2020-11-20T13:50:26.446 INFO:teuthology.orchestra.run.smithi085:> hostname 2020-11-20T13:50:26.464 INFO:teuthology.orchestra.run.smithi085.stdout:smithi085.front.sepia.ceph.com 2020-11-20T13:50:26.465 INFO:teuthology.orchestra.run.smithi085:> grep smithi085.front.sepia.ceph.com /etc/hosts 2020-11-20T13:50:26.521 INFO:teuthology.orchestra.run.smithi085.stdout:172.21.15.85 smithi085.front.sepia.ceph.com smithi085 2020-11-20T13:50:26.521 INFO:teuthology.orchestra.run.smithi085:> sudo hostname smithi085 2020-11-20T13:50:26.650 INFO:teuthology.orchestra.run.smithi085:> sudo sed -i -e 's/smithi085.front.sepia.ceph.com/smithi085/g' /etc/hosts 2020-11-20T13:50:26.680 INFO:teuthology.orchestra.run.smithi085:> sudo sed -i -e 's/172.21.15.85/172.21.15.85/g' /etc/hosts 2020-11-20T13:50:26.749 INFO:teuthology.orchestra.run.smithi085:> sudo sed -i -e 's/smithi085.front.sepia.ceph.com/smithi085/g' /etc/hostname 2020-11-20T13:50:26.818 INFO:teuthology.orchestra.run.smithi085:> sudo hostname smithi085 2020-11-20T13:50:26.884 INFO:teuthology.provision.fog.smithi085:Deploy complete! 2020-11-20T13:50:26.885 DEBUG:teuthology.parallel:result is None 2020-11-20T13:50:43.708 INFO:teuthology.provision.fog.smithi117:Node is ready 2020-11-20T13:50:43.708 INFO:teuthology.orchestra.run.smithi117:> hostname 2020-11-20T13:50:43.726 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117.front.sepia.ceph.com 2020-11-20T13:50:43.726 INFO:teuthology.orchestra.run.smithi117:> grep smithi117.front.sepia.ceph.com /etc/hosts 2020-11-20T13:50:43.782 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117 smithi117.front.sepia.ceph.com smithi117 2020-11-20T13:50:43.783 INFO:teuthology.orchestra.run.smithi117:> sudo hostname smithi117 2020-11-20T13:50:43.930 INFO:teuthology.orchestra.run.smithi117:> sudo sed -i -e 's/smithi117.front.sepia.ceph.com/smithi117/g' /etc/hosts 2020-11-20T13:50:43.960 INFO:teuthology.orchestra.run.smithi117:> sudo sed -i -e 's/172.21.15.117/172.21.15.117/g' /etc/hosts 2020-11-20T13:50:44.029 INFO:teuthology.orchestra.run.smithi117:> sudo sed -i -e 's/smithi117.front.sepia.ceph.com/smithi117/g' /etc/hostname 2020-11-20T13:50:44.098 INFO:teuthology.orchestra.run.smithi117:> sudo hostname smithi117 2020-11-20T13:50:44.165 INFO:teuthology.provision.fog.smithi117:Deploy complete! 2020-11-20T13:50:44.166 DEBUG:teuthology.parallel:result is None 2020-11-20T13:50:44.296 INFO:teuthology.lock.ops:Checking smithi117.front.sepia.ceph.com 2020-11-20T13:50:44.297 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T13:50:44.313 INFO:teuthology.lock.ops:Checking smithi085.front.sepia.ceph.com 2020-11-20T13:50:44.314 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T13:50:44.357 INFO:teuthology.lock.ops:Updating [smithi117.front.sepia.ceph.com]: set os type and version on server 2020-11-20T13:50:44.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:50:44.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:44.423 INFO:teuthology.orchestra.run.smithi117:> true 2020-11-20T13:50:44.578 INFO:teuthology.orchestra.run.smithi117:> uname -m 2020-11-20T13:50:44.631 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2020-11-20T13:50:44.632 INFO:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2020-11-20T13:50:44.684 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Linux" 2020-11-20T13:50:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="8 (Core)" 2020-11-20T13:50:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2020-11-20T13:50:44.685 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:50:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="8" 2020-11-20T13:50:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:50:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:50:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2020-11-20T13:50:44.686 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:50:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:50:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:50:44.687 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:50:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:50:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:50:44.687 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:50:44.688 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:50:44.688 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:50:44.689 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2020-11-20T13:50:44.706 INFO:teuthology.lock.ops:Updating [smithi085.front.sepia.ceph.com]: set os type and version on server 2020-11-20T13:50:44.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:50:44.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:44.775 INFO:teuthology.orchestra.run.smithi085:> true 2020-11-20T13:50:44.944 INFO:teuthology.orchestra.run.smithi085:> uname -m 2020-11-20T13:50:45.002 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2020-11-20T13:50:45.002 INFO:teuthology.orchestra.run.smithi085:> cat /etc/os-release 2020-11-20T13:50:45.061 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="CentOS Linux" 2020-11-20T13:50:45.062 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="8 (Core)" 2020-11-20T13:50:45.062 INFO:teuthology.orchestra.run.smithi085.stdout:ID="centos" 2020-11-20T13:50:45.062 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:50:45.062 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="8" 2020-11-20T13:50:45.062 INFO:teuthology.orchestra.run.smithi085.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:50:45.063 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:50:45.063 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31" 2020-11-20T13:50:45.063 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:50:45.063 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:50:45.063 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:50:45.064 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:50:45.064 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:50:45.064 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:50:45.064 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:50:45.064 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:50:45.065 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:50:45.065 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2020-11-20T13:50:45.082 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-11-20T13:50:45.127 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkpy7YtpzpBoIaad9dS53i37YxrKO6patHbk4vlp7vUFrs7QTpFC5a7hx+Tr+P7UTTtPXgbEurBYgJZXK3mrOh6qPzG0XbKNqCPTFgTEpIsJkZ8a2bTD9o1qijJ+bf7eust/cyhAkHQdalKUhAvzVx2OruYtRWRrsLCh+NszdDIAJ7iiyd45T25xTEnGUt5YKbJtxbqUVnSuzYe9Je0930MuB6S8NBwSw3IoDdiZX9Z4oo57B52nrDK8OdWmTLrTvkQAT+g40HSNIY+cag+gkMzYWtb3AxblMAD+NHyPa2AYTXNj8zYXNaD1EpqBrnKTkeDzNAO+728u9M1hmQbJFdjU5+/dIoPNypUxnIFXl66oj1OS0y2dUrc9LKCnS5IhLmIpOu492E4Tk71qxR5KckJres43zvHVx2ehD9kEmcN/11RniofGG5VCnCAGifuo0WpPYz3p+kw4a19Sd/0X/VikK+WK9Vz5N7ul1amAQ55HDHcfTLMZ/3mty1Fpnx6/s= smithi117.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtDP5HjQgDih0wZKqRJqB7oDZLctFdP4o1P05xVnYWarQ1t61y1CZdNvDNjHoP+ixSrDpXbzcV0TWfkAlwYDBMYjwrMT27LrzFkjdStdDHU7E4lo9KbK4Mgu8QkkDbn92Xj46N/cWUiFNR4EVpw7G34Cu04NBFLw/aiX/mFCCsPD+ZsFnWRf45qrtHGjxtLM9p4dzIInjQ6zqpeODxq7JQKCBlU14lT9BVqeJVa1ZwqT5ShIU0cKkfC7prTyH9HNDCFnTWWqUcz52swGHB0MhavQZwwR9dnoUfOBCJXfIPzdj5A7qBFxFGGKpt/8/34T94/dmyAykmRCRdMjkAPb/ODlev89dCl0PTlQVhkMQj/DFdtaFjwdCeFY+xwpk9xKBKYfT4/sjI7YU6HS73CRa0qUM5nlBBB3mVDt7ZCn151Zb0RfDJbXSg0oEHxZHY26auIpd9DOZ5TSO5Yki3Jr5hFW93nOfn3mQKypa+h5Pv1clEF6xWv4pf8LTWHDlgcOs= 2020-11-20T13:50:45.127 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T13:50:45.194 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-20T13:50:45.201 INFO:teuthology.task.internal:Saving configuration 2020-11-20T13:50:45.230 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-20T13:50:45.236 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-20T13:50:45.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 13:45:51.286316', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628501', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN5xjNXqtFtDWQRDKDMegXL9OY2wzoocSrj0J4oDDPSi/DsM8GXqV6EcRucFGegj7dfDjkYc8oW9els83eKJTuY=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi117.front.sepia.ceph.com'} 2020-11-20T13:50:45.286 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 13:45:51.286337', 'description': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628501', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGUeFfrCtQabaSPn+YtEfSDtEQPsng+ep8Zq5jxyDjlt918+BL2PrSAk72gnrF6m5DdBSBz3wVBym3Z88nahwc=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi085.front.sepia.ceph.com'} 2020-11-20T13:50:45.286 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-20T13:50:45.294 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'] 2020-11-20T13:50:45.295 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3'] 2020-11-20T13:50:45.295 INFO:teuthology.run_tasks:Running task console_log... 2020-11-20T13:50:45.449 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3590654598>, signals=[15]) 2020-11-20T13:50:45.450 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-20T13:50:45.465 INFO:teuthology.task.internal:Opening connections... 2020-11-20T13:50:45.465 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2020-11-20T13:50:45.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:45.549 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi085.front.sepia.ceph.com 2020-11-20T13:50:45.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:50:45.672 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-20T13:50:45.682 INFO:teuthology.orchestra.run.smithi117:> uname -m 2020-11-20T13:50:45.697 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2020-11-20T13:50:45.698 INFO:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2020-11-20T13:50:45.751 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Linux" 2020-11-20T13:50:45.751 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="8 (Core)" 2020-11-20T13:50:45.751 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2020-11-20T13:50:45.752 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:50:45.752 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="8" 2020-11-20T13:50:45.752 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:50:45.752 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:50:45.752 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2020-11-20T13:50:45.753 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:50:45.753 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:50:45.753 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:50:45.753 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:50:45.753 INFO:teuthology.orchestra.run.smithi117.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:50:45.754 INFO:teuthology.orchestra.run.smithi117.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:50:45.754 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:50:45.754 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:50:45.754 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:50:45.755 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2020-11-20T13:50:45.774 INFO:teuthology.orchestra.run.smithi085:> uname -m 2020-11-20T13:50:45.789 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2020-11-20T13:50:45.789 INFO:teuthology.orchestra.run.smithi085:> cat /etc/os-release 2020-11-20T13:50:45.842 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="CentOS Linux" 2020-11-20T13:50:45.842 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="8 (Core)" 2020-11-20T13:50:45.843 INFO:teuthology.orchestra.run.smithi085.stdout:ID="centos" 2020-11-20T13:50:45.843 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="rhel fedora" 2020-11-20T13:50:45.843 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="8" 2020-11-20T13:50:45.843 INFO:teuthology.orchestra.run.smithi085.stdout:PLATFORM_ID="platform:el8" 2020-11-20T13:50:45.843 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-11-20T13:50:45.844 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31" 2020-11-20T13:50:45.844 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-11-20T13:50:45.844 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.centos.org/" 2020-11-20T13:50:45.844 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-11-20T13:50:45.844 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:50:45.845 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-11-20T13:50:45.845 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-11-20T13:50:45.845 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-11-20T13:50:45.845 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-11-20T13:50:45.845 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:50:45.846 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2020-11-20T13:50:45.861 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-20T13:50:45.873 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-20T13:50:45.879 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-20T13:50:45.880 INFO:teuthology.orchestra.run.smithi085:> test '!' -e /home/ubuntu/cephtest 2020-11-20T13:50:45.884 INFO:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2020-11-20T13:50:45.897 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-20T13:50:45.906 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-20T13:50:45.906 INFO:teuthology.orchestra.run.smithi085:> test -z $(ls -A /var/lib/ceph) 2020-11-20T13:50:45.937 INFO:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2020-11-20T13:50:45.959 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-20T13:50:46.021 INFO:teuthology.run_tasks:Running task kernel... 2020-11-20T13:50:46.112 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}, 'client.3': {'kdb': True, 'sha1': 'distro'}} 2020-11-20T13:50:46.113 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-11-20T13:50:46.113 INFO:teuthology.orchestra.run.smithi117:> uname -r 2020-11-20T13:50:46.126 INFO:teuthology.orchestra.run.smithi117.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:50:46.126 INFO:teuthology.task.kernel:Running kernel on smithi117: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:50:46.127 INFO:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2020-11-20T13:50:48.608 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-8 - AppStream 39 MB/s | 7.0 MB 00:00 2020-11-20T13:50:49.803 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-11-20T13:50:50.229 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-8 - Extras 87 kB/s | 5.5 kB 00:00 2020-11-20T13:50:50.729 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 18 MB/s | 8.3 MB 00:00 2020-11-20T13:50:52.460 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 754 kB/s | 23 kB 00:00 2020-11-20T13:50:53.420 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-20T13:50:53.420 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-20T13:50:53.590 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2020-11-20T13:50:53.591 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2020-11-20T13:50:53.591 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2020-11-20T13:50:53.710 INFO:teuthology.orchestra.run.smithi117:> echo no | sudo yum reinstall kernel || true 2020-11-20T13:50:54.116 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 01:50:52 PM UTC. 2020-11-20T13:50:54.362 INFO:teuthology.orchestra.run.smithi117.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:50:54.362 INFO:teuthology.orchestra.run.smithi117.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:50:54.410 INFO:teuthology.orchestra.run.smithi117.stderr:Error: No packages marked for reinstall. 2020-11-20T13:50:54.454 INFO:teuthology.orchestra.run.smithi117:> sudo yum reinstall -y kernel || true 2020-11-20T13:50:54.855 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 01:50:52 PM UTC. 2020-11-20T13:50:55.111 INFO:teuthology.orchestra.run.smithi117.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:50:55.112 INFO:teuthology.orchestra.run.smithi117.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:50:55.143 INFO:teuthology.orchestra.run.smithi117.stderr:Error: No packages marked for reinstall. 2020-11-20T13:50:55.183 INFO:teuthology.orchestra.run.smithi117:> rpm -q kernel --last | head -n 1 2020-11-20T13:50:55.241 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-20T13:50:55.242 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:50:55.243 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-20T13:50:55.243 INFO:teuthology.orchestra.run.smithi085:> uname -r 2020-11-20T13:50:55.260 INFO:teuthology.orchestra.run.smithi085.stdout:4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:50:55.260 INFO:teuthology.task.kernel:Running kernel on smithi085: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:50:55.260 INFO:teuthology.orchestra.run.smithi085:> sudo yum install -y kernel 2020-11-20T13:50:57.828 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS-8 - AppStream 41 MB/s | 7.0 MB 00:00 2020-11-20T13:50:59.050 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS-8 - Base 23 MB/s | 2.2 MB 00:00 2020-11-20T13:50:59.478 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS-8 - Extras 87 kB/s | 5.5 kB 00:00 2020-11-20T13:50:59.969 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 19 MB/s | 8.3 MB 00:00 2020-11-20T13:51:01.710 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 3.8 MB/s | 23 kB 00:00 2020-11-20T13:51:02.631 INFO:teuthology.orchestra.run.smithi085.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-11-20T13:51:02.631 INFO:teuthology.orchestra.run.smithi085.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2020-11-20T13:51:02.793 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2020-11-20T13:51:02.794 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2020-11-20T13:51:02.794 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2020-11-20T13:51:02.928 INFO:teuthology.orchestra.run.smithi085:> echo no | sudo yum reinstall kernel || true 2020-11-20T13:51:03.331 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Nov 2020 01:51:01 PM UTC. 2020-11-20T13:51:03.575 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:51:03.575 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:51:03.652 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2020-11-20T13:51:03.703 INFO:teuthology.orchestra.run.smithi085:> sudo yum reinstall -y kernel || true 2020-11-20T13:51:04.104 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Nov 2020 01:51:01 PM UTC. 2020-11-20T13:51:04.358 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-11-20T13:51:04.359 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-193.28.1.el8_2.x86_64 (from BaseOS) not available. 2020-11-20T13:51:04.402 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2020-11-20T13:51:04.449 INFO:teuthology.orchestra.run.smithi085:> rpm -q kernel --last | head -n 1 2020-11-20T13:51:04.511 INFO:teuthology.orchestra.run.smithi085.stdout:kernel-4.18.0-193.28.1.el8_2.x86_64 Sun 15 Nov 2020 12:12:26 PM UTC 2020-11-20T13:51:04.513 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.28.1.el8_2.x86_64 2020-11-20T13:51:04.513 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-11-20T13:51:04.513 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-11-20T13:51:04.514 INFO:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-20T13:51:04.555 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2020-11-20T13:51:04.570 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-11-20T13:51:04.570 INFO:teuthology.orchestra.run.smithi085:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-20T13:51:04.613 INFO:teuthology.orchestra.run.smithi085.stdout:ttyS1 2020-11-20T13:51:04.624 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-20T13:51:04.636 INFO:teuthology.task.internal:Creating test directory... 2020-11-20T13:51:04.636 INFO:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T13:51:04.666 INFO:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T13:51:04.682 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-20T13:51:04.693 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-20T13:51:04.775 INFO:teuthology.task.internal:Creating archive directory... 2020-11-20T13:51:04.776 INFO:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T13:51:04.778 INFO:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T13:51:04.803 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-20T13:51:04.815 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-20T13:51:04.815 INFO:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T13:51:04.841 INFO:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T13:51:04.884 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:04.899 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:04.907 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:04.920 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T13:51:04.921 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-20T13:51:04.929 INFO:teuthology.task.internal:Configuring sudo... 2020-11-20T13:51:04.929 INFO:teuthology.orchestra.run.smithi085:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T13:51:04.931 INFO:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T13:51:04.970 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-20T13:51:04.979 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-20T13:51:04.979 INFO:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T13:51:04.998 INFO:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T13:51:05.025 INFO:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:05.101 INFO:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:05.159 INFO:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:05.232 INFO:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:05.284 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:51:05.284 INFO:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:05.351 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2020-11-20T13:51:05.352 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2020-11-20T13:51:05.352 INFO:teuthology.orchestra.run.smithi117.stderr:152 bytes copied, 3.2028e-05 s, 4.7 MB/s 2020-11-20T13:51:05.353 INFO:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:05.386 INFO:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T13:51:05.446 INFO:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:05.517 INFO:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T13:51:05.568 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:51:05.568 INFO:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:05.621 INFO:teuthology.orchestra.run.smithi085.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T13:51:05.633 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records in 2020-11-20T13:51:05.633 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records out 2020-11-20T13:51:05.633 INFO:teuthology.orchestra.run.smithi085.stderr:152 bytes copied, 2.9897e-05 s, 5.1 MB/s 2020-11-20T13:51:05.634 INFO:teuthology.orchestra.run.smithi085:> sudo service rsyslog restart 2020-11-20T13:51:05.676 INFO:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2020-11-20T13:51:05.742 INFO:teuthology.orchestra.run.smithi085.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-20T13:51:05.760 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-11-20T13:51:06.182 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-20T13:51:06.191 INFO:teuthology.task.internal:Starting timer... 2020-11-20T13:51:06.191 INFO:teuthology.run_tasks:Running task pcp... 2020-11-20T13:51:06.217 INFO:teuthology.run_tasks:Running task selinux... 2020-11-20T13:51:06.230 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-11-20T13:51:06.231 INFO:teuthology.orchestra.run.smithi085:> sudo service auditd rotate 2020-11-20T13:51:06.288 INFO:teuthology.orchestra.run.smithi085.stdout:Rotating logs: [ OK ] 2020-11-20T13:51:06.290 INFO:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2020-11-20T13:51:06.348 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: [ OK ] 2020-11-20T13:51:06.350 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-20T13:51:06.350 INFO:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2020-11-20T13:51:06.370 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2020-11-20T13:51:06.370 INFO:teuthology.orchestra.run.smithi085:> /usr/sbin/getenforce 2020-11-20T13:51:06.397 INFO:teuthology.orchestra.run.smithi085.stdout:Permissive 2020-11-20T13:51:06.397 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi085.front.sepia.ceph.com': 'permissive'} 2020-11-20T13:51:06.397 INFO:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-20T13:51:06.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:51:06.439 INFO:teuthology.orchestra.run.smithi085:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-11-20T13:51:06.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:51:06.467 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-20T13:51:06.467 INFO:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2020-11-20T13:51:06.508 INFO:teuthology.orchestra.run.smithi085:> sudo /usr/sbin/setenforce permissive 2020-11-20T13:51:06.536 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-20T13:51:06.545 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2020-11-20T13:51:06.551 INFO:teuthology.repo_utils:Fetching master from origin 2020-11-20T13:51:06.610 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-20T13:51:06.621 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-20T13:51:06.622 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi117.front.sepia.ceph.com,smithi085.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-20T13:51:06.908 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-20T13:51:07.671 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-20T13:51:09.891 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-20T13:51:09.908 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:09 +0000 (0:00:00.060) 0:00:00.060 ******* 2020-11-20T13:51:09.920 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:09 +0000 (0:00:00.011) 0:00:00.072 ******* 2020-11-20T13:51:11.297 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi085.front.sepia.ceph.com] 2020-11-20T13:51:11.360 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:11 +0000 (0:00:01.440) 0:00:01.512 ******* 2020-11-20T13:51:11.655 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-20T13:51:11.657 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:51:11.687 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:11 +0000 (0:00:00.326) 0:00:01.839 ******* 2020-11-20T13:51:12.053 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-20T13:51:12.054 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-20T13:51:12.085 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:12 +0000 (0:00:00.398) 0:00:02.237 ******* 2020-11-20T13:51:12.110 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-20T13:51:12.112 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:12.142 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:12 +0000 (0:00:00.057) 0:00:02.294 ******* 2020-11-20T13:51:12.348 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi117.front.sepia.ceph.com] 2020-11-20T13:51:12.417 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:12 +0000 (0:00:00.274) 0:00:02.569 ******* 2020-11-20T13:51:12.433 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-20T13:51:12.435 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:12.470 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:12 +0000 (0:00:00.052) 0:00:02.622 ******* 2020-11-20T13:51:12.600 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-20T13:51:12.601 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:51:12.633 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:12 +0000 (0:00:00.162) 0:00:02.785 ******* 2020-11-20T13:51:12.855 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-20T13:51:12.857 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-20T13:51:12.887 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:12 +0000 (0:00:00.254) 0:00:03.039 ******* 2020-11-20T13:51:12.912 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-20T13:51:12.914 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:12.944 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:12 +0000 (0:00:00.057) 0:00:03.096 ******* 2020-11-20T13:51:13.127 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-20T13:51:13.128 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:13.157 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:13 +0000 (0:00:00.212) 0:00:03.309 ******* 2020-11-20T13:51:13.358 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-20T13:51:13.359 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880272.4817216-205414157837768/source", "state": "file", "uid": 0} 2020-11-20T13:51:13.389 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:13 +0000 (0:00:00.232) 0:00:03.541 ******* 2020-11-20T13:51:13.715 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880273.1681743-245373170448603/source", "state": "file", "uid": 0} 2020-11-20T13:51:13.748 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:13 +0000 (0:00:00.358) 0:00:03.900 ******* 2020-11-20T13:51:13.783 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-20T13:51:13.785 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T13:51:13.787 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T13:51:13.995 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T13:51:13.996 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T13:51:14.079 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-11-20T13:51:14.085 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-11-20T13:51:14.211 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:14 +0000 (0:00:00.462) 0:00:04.363 ******* 2020-11-20T13:51:14.237 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:14 +0000 (0:00:00.025) 0:00:04.388 ******* 2020-11-20T13:51:14.836 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi085.front.sepia.ceph.com] 2020-11-20T13:51:14.874 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] 2020-11-20T13:51:14.913 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-20T13:51:14.914 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:14 +0000 (0:00:00.676) 0:00:05.065 ******* 2020-11-20T13:51:14.971 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:14 +0000 (0:00:00.058) 0:00:05.123 ******* 2020-11-20T13:51:15.335 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-20T13:51:15.338 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.146785", "end": "2020-11-20 13:51:15.312622", "rc": 0, "start": "2020-11-20 13:51:15.165837", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} changed: [smithi085.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.153813", "end": "2020-11-20 13:51:15.312090", "rc": 0, "start": "2020-11-20 13:51:15.158277", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T13:51:15.366 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.394) 0:00:05.518 ******* 2020-11-20T13:51:15.393 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.026) 0:00:05.544 ******* 2020-11-20T13:51:15.562 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-20T13:51:15.563 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.006154", "end": "2020-11-20 13:51:15.536943", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:51:15.530789", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T13:51:15.593 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.200) 0:00:05.745 ******* 2020-11-20T13:51:15.598 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008379", "end": "2020-11-20 13:51:15.573310", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:51:15.564931", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T13:51:15.629 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.035) 0:00:05.781 ******* 2020-11-20T13:51:15.633 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-20T13:51:15.635 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-20T13:51:15.664 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.034) 0:00:05.816 ******* 2020-11-20T13:51:15.668 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-20T13:51:15.700 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.036) 0:00:05.852 ******* 2020-11-20T13:51:15.704 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-20T13:51:15.706 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.734 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.034) 0:00:05.886 ******* 2020-11-20T13:51:15.738 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.769 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.034) 0:00:05.921 ******* 2020-11-20T13:51:15.773 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-20T13:51:15.775 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.804 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.035) 0:00:05.956 ******* 2020-11-20T13:51:15.809 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.839 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.034) 0:00:05.991 ******* 2020-11-20T13:51:15.843 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-20T13:51:15.844 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.876 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.036) 0:00:06.028 ******* 2020-11-20T13:51:15.881 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.913 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.036) 0:00:06.065 ******* 2020-11-20T13:51:15.917 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-20T13:51:15.918 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.948 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.035) 0:00:06.100 ******* 2020-11-20T13:51:15.953 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:15.984 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:15 +0000 (0:00:00.035) 0:00:06.136 ******* 2020-11-20T13:51:15.989 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-20T13:51:15.991 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.021 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:16 +0000 (0:00:00.037) 0:00:06.173 ******* 2020-11-20T13:51:16.027 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.060 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:16 +0000 (0:00:00.038) 0:00:06.212 ******* 2020-11-20T13:51:16.066 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-20T13:51:16.068 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.078 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.080 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr 2020-11-20T13:51:16.081 INFO:teuthology.task.ansible.out:+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.090 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.092 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.101 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.103 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.113 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.115 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBH 2020-11-20T13:51:16.117 INFO:teuthology.task.ansible.out:WOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.125 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.127 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22 2020-11-20T13:51:16.129 INFO:teuthology.task.ansible.out:c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.138 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.139 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef 2020-11-20T13:51:16.141 INFO:teuthology.task.ansible.out:8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.151 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.152 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQ 2020-11-20T13:51:16.154 INFO:teuthology.task.ansible.out:ABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.163 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.164 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n' 2020-11-20T13:51:16.166 INFO:teuthology.task.ansible.out:}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.175 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.176 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.177 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXY 2020-11-20T13:51:16.179 INFO:teuthology.task.ansible.out:UFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.189 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.191 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was Fa 2020-11-20T13:51:16.192 INFO:teuthology.task.ansible.out:lse"} 2020-11-20T13:51:16.201 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.203 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.213 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.214 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.216 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.226 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.228 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.238 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.240 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.240 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086c 2020-11-20T13:51:16.242 INFO:teuthology.task.ansible.out:e668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.251 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.252 INFO:teuthology.task.ansible.out: skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.254 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.254 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.256 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.266 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.267 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.278 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.279 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.290 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.292 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.303 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.305 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.305 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.316 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.317 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.318 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.328 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.330 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.330 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmoha 2020-11-20T13:51:16.332 INFO:teuthology.task.ansible.out:n@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.341 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.342 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.353 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.355 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.355 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.355 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item" 2020-11-20T13:51:16.357 INFO:teuthology.task.ansible.out:: {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.366 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.368 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.378 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.379 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.379 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.381 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.391 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.392 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.393 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.403 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.405 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.415 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.416 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.416 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.427 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.428 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.439 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.440 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.451 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.453 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.453 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.464 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.465 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.466 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.476 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.478 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.478 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.489 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.491 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.491 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'skra 2020-11-20T13:51:16.493 INFO:teuthology.task.ansible.out:h', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.502 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.504 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.514 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.515 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.516 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.526 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.528 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.529 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'yl 2020-11-20T13:51:16.530 INFO:teuthology.task.ansible.out:ifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.540 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.541 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.552 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.553 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.554 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.564 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.566 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost 2020-11-20T13:51:16.568 INFO:teuthology.task.ansible.out:IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.577 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.578 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.578 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.580 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.590 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.592 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.592 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.603 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.604 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.615 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.616 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.617 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.628 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.629 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.630 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.640 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.641 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.642 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.643 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.653 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.655 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.657 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.666 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.668 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.668 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.679 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.680 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.681 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.692 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.692 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.693 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.694 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.705 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.706 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.707 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.708 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.718 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.719 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.720 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.720 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.731 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.732 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.743 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.745 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.745 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.745 INFO:teuthology.task.ansible.out:skipping: [smithi085 2020-11-20T13:51:16.747 INFO:teuthology.task.ansible.out:.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.756 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.758 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.774 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.775 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.809 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:16 +0000 (0:00:00.748) 0:00:06.961 ******* 2020-11-20T13:51:16.815 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.816 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.818 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.819 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.821 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.838 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-20T13:51:16.871 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:16 +0000 (0:00:00.061) 0:00:07.023 ******* 2020-11-20T13:51:16.900 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:16 +0000 (0:00:00.028) 0:00:07.051 ******* 2020-11-20T13:51:16.905 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:16.942 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:16 +0000 (0:00:00.042) 0:00:07.094 ******* 2020-11-20T13:51:16.947 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-20T13:51:16.978 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:16 +0000 (0:00:00.035) 0:00:07.130 ******* 2020-11-20T13:51:16.983 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-20T13:51:16.984 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.016 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.037) 0:00:07.168 ******* 2020-11-20T13:51:17.021 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-20T13:51:17.022 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.058 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.041) 0:00:07.210 ******* 2020-11-20T13:51:17.063 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-20T13:51:17.065 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.098 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.039) 0:00:07.250 ******* 2020-11-20T13:51:17.103 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-20T13:51:17.105 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.136 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.037) 0:00:07.288 ******* 2020-11-20T13:51:17.140 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-20T13:51:17.141 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.171 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.035) 0:00:07.323 ******* 2020-11-20T13:51:17.175 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-20T13:51:17.177 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.207 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.035) 0:00:07.359 ******* 2020-11-20T13:51:17.211 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-20T13:51:17.213 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.244 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.036) 0:00:07.396 ******* 2020-11-20T13:51:17.249 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-20T13:51:17.251 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.286 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.041) 0:00:07.438 ******* 2020-11-20T13:51:17.292 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-20T13:51:17.293 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.294 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.305 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.306 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2020-11-20T13:51:17.306 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.337 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.051) 0:00:07.489 ******* 2020-11-20T13:51:17.342 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.343 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.343 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.344 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.344 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.344 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.345 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.356 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.357 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.368 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.369 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.379 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.381 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.391 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.392 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.392 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.393 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.393 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.393 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.404 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.406 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.416 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.418 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.419 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.429 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.429 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.430 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.430 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.431 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.441 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.443 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.454 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.455 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.456 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.466 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.468 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.478 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.480 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.480 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.480 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.481 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.491 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.495 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.497 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.530 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.192) 0:00:07.682 ******* 2020-11-20T13:51:17.535 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.537 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.537 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.539 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.539 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.539 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.550 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.552 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.562 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.563 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.573 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.574 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.585 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.586 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:17.625 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:17 +0000 (0:00:00.095) 0:00:07.777 ******* 2020-11-20T13:51:18.219 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-20T13:51:18.221 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-11-20T13:51:18.253 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.627) 0:00:08.405 ******* 2020-11-20T13:51:18.282 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.028) 0:00:08.434 ******* 2020-11-20T13:51:18.286 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-20T13:51:18.288 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => { "msg": "Host smithi085.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-20T13:51:18.320 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.038) 0:00:08.472 ******* 2020-11-20T13:51:18.325 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => { "msg": "Host smithi117.front.sepia.ceph.com is running CentOS 8.1 (Core)" } 2020-11-20T13:51:18.359 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.038) 0:00:08.511 ******* 2020-11-20T13:51:18.365 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-20T13:51:18.367 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/{{ container_mirror }}", "nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-20T13:51:18.398 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.039) 0:00:08.550 ******* 2020-11-20T13:51:18.404 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/{{ container_mirror }}", "nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2020-11-20T13:51:18.438 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.039) 0:00:08.590 ******* 2020-11-20T13:51:18.442 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-20T13:51:18.444 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:18.474 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.036) 0:00:08.626 ******* 2020-11-20T13:51:18.479 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:18.513 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:18 +0000 (0:00:00.038) 0:00:08.665 ******* 2020-11-20T13:51:18.994 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-20T13:51:18.996 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.336900", "end": "2020-11-20 13:51:18.974017", "rc": 0, "start": "2020-11-20 13:51:18.637117", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T13:51:19.027 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.514) 0:00:09.179 ******* 2020-11-20T13:51:19.033 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.311971", "end": "2020-11-20 13:51:18.992375", "rc": 0, "start": "2020-11-20 13:51:18.680404", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T13:51:19.066 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.218 ******* 2020-11-20T13:51:19.071 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-20T13:51:19.072 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.106 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.258 ******* 2020-11-20T13:51:19.112 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.146 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.040) 0:00:09.298 ******* 2020-11-20T13:51:19.151 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-20T13:51:19.153 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.186 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.338 ******* 2020-11-20T13:51:19.190 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.222 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.036) 0:00:09.374 ******* 2020-11-20T13:51:19.228 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-20T13:51:19.230 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.260 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.037) 0:00:09.412 ******* 2020-11-20T13:51:19.264 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.298 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.450 ******* 2020-11-20T13:51:19.303 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-20T13:51:19.304 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.337 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.489 ******* 2020-11-20T13:51:19.342 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.376 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.528 ******* 2020-11-20T13:51:19.382 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-20T13:51:19.383 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.416 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.568 ******* 2020-11-20T13:51:19.421 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.454 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.606 ******* 2020-11-20T13:51:19.459 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-20T13:51:19.460 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.494 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.646 ******* 2020-11-20T13:51:19.499 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.533 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.685 ******* 2020-11-20T13:51:19.539 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-20T13:51:19.540 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi085.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:19.572 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.724 ******* 2020-11-20T13:51:19.577 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:19.579 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:19.610 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.762 ******* 2020-11-20T13:51:19.615 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-20T13:51:19.617 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.801 ******* 2020-11-20T13:51:19.654 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.687 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.839 ******* 2020-11-20T13:51:19.692 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-20T13:51:19.694 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.726 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:09.878 ******* 2020-11-20T13:51:19.732 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.765 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.917 ******* 2020-11-20T13:51:19.771 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-20T13:51:19.772 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:19.804 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.039) 0:00:09.956 ******* 2020-11-20T13:51:19.810 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:19.842 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.037) 0:00:09.994 ******* 2020-11-20T13:51:19.847 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-20T13:51:19.849 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.881 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:10.033 ******* 2020-11-20T13:51:19.886 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:19.923 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.041) 0:00:10.075 ******* 2020-11-20T13:51:19.928 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-20T13:51:19.929 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:19.962 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:10.114 ******* 2020-11-20T13:51:19.967 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:20.000 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:19 +0000 (0:00:00.038) 0:00:10.152 ******* 2020-11-20T13:51:20.005 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-20T13:51:20.006 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.038 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:10.190 ******* 2020-11-20T13:51:20.043 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.077 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.039) 0:00:10.229 ******* 2020-11-20T13:51:20.083 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-20T13:51:20.084 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.116 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.038) 0:00:10.268 ******* 2020-11-20T13:51:20.121 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.154 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:10.306 ******* 2020-11-20T13:51:20.159 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-20T13:51:20.160 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.192 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.038) 0:00:10.344 ******* 2020-11-20T13:51:20.198 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.229 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.036) 0:00:10.381 ******* 2020-11-20T13:51:20.234 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-20T13:51:20.235 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.267 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:10.419 ******* 2020-11-20T13:51:20.272 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.305 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.038) 0:00:10.457 ******* 2020-11-20T13:51:20.310 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-20T13:51:20.311 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:20.341 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.035) 0:00:10.493 ******* 2020-11-20T13:51:20.346 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:20.380 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.039) 0:00:10.532 ******* 2020-11-20T13:51:20.385 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T13:51:20.387 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.418 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.038) 0:00:10.570 ******* 2020-11-20T13:51:20.425 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.457 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.038) 0:00:10.609 ******* 2020-11-20T13:51:20.462 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-20T13:51:20.463 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.495 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:10.647 ******* 2020-11-20T13:51:20.500 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.531 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.036) 0:00:10.683 ******* 2020-11-20T13:51:20.536 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-20T13:51:20.537 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.568 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.036) 0:00:10.720 ******* 2020-11-20T13:51:20.573 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.605 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.036) 0:00:10.757 ******* 2020-11-20T13:51:20.609 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-20T13:51:20.611 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.642 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:10.794 ******* 2020-11-20T13:51:20.648 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.683 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.040) 0:00:10.835 ******* 2020-11-20T13:51:20.688 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-20T13:51:20.689 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.722 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.039) 0:00:10.874 ******* 2020-11-20T13:51:20.727 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.762 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.039) 0:00:10.914 ******* 2020-11-20T13:51:20.766 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-20T13:51:20.768 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.800 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.038) 0:00:10.952 ******* 2020-11-20T13:51:20.806 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.838 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:10.990 ******* 2020-11-20T13:51:20.843 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-20T13:51:20.844 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:20.875 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:11.027 ******* 2020-11-20T13:51:20.881 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:51:20.913 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.038) 0:00:11.065 ******* 2020-11-20T13:51:20.918 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-20T13:51:20.919 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.951 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:11.103 ******* 2020-11-20T13:51:20.957 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:20.989 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:20 +0000 (0:00:00.037) 0:00:11.141 ******* 2020-11-20T13:51:20.994 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-20T13:51:20.996 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:21.029 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:21 +0000 (0:00:00.039) 0:00:11.181 ******* 2020-11-20T13:51:21.034 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:21.068 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:21 +0000 (0:00:00.039) 0:00:11.220 ******* 2020-11-20T13:51:21.229 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-20T13:51:21.231 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:21.263 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:21 +0000 (0:00:00.194) 0:00:11.415 ******* 2020-11-20T13:51:21.279 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:21.313 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:21 +0000 (0:00:00.049) 0:00:11.465 ******* 2020-11-20T13:51:21.711 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-20T13:51:21.713 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-20T13:51:21.764 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-11-20T13:51:22.139 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-20T13:51:22.173 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:22 +0000 (0:00:00.859) 0:00:12.325 ******* 2020-11-20T13:51:22.179 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-11-20T13:51:22.216 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:22 +0000 (0:00:00.043) 0:00:12.368 ******* 2020-11-20T13:51:22.631 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-20T13:51:22.633 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-20T13:51:22.684 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-11-20T13:51:23.057 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-20T13:51:23.093 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.876) 0:00:13.245 ******* 2020-11-20T13:51:23.118 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T13:51:23.120 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi117.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-11-20T13:51:23.154 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.060) 0:00:13.306 ******* 2020-11-20T13:51:23.182 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.028) 0:00:13.334 ******* 2020-11-20T13:51:23.188 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:23.189 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.221 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.039) 0:00:13.373 ******* 2020-11-20T13:51:23.227 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T13:51:23.228 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.258 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.036) 0:00:13.410 ******* 2020-11-20T13:51:23.263 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-20T13:51:23.264 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.037) 0:00:13.448 ******* 2020-11-20T13:51:23.301 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:23.303 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.334 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.037) 0:00:13.486 ******* 2020-11-20T13:51:23.340 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-20T13:51:23.342 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.372 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.038) 0:00:13.524 ******* 2020-11-20T13:51:23.377 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-20T13:51:23.379 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.037) 0:00:13.562 ******* 2020-11-20T13:51:23.414 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:23.416 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.447 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.037) 0:00:13.599 ******* 2020-11-20T13:51:23.452 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-20T13:51:23.453 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.484 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.037) 0:00:13.636 ******* 2020-11-20T13:51:23.489 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-20T13:51:23.490 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.522 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.037) 0:00:13.674 ******* 2020-11-20T13:51:23.527 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T13:51:23.529 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.560 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.037) 0:00:13.712 ******* 2020-11-20T13:51:23.565 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-20T13:51:23.566 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.598 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.038) 0:00:13.750 ******* 2020-11-20T13:51:23.604 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-20T13:51:23.605 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:23.637 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:23 +0000 (0:00:00.038) 0:00:13.789 ******* 2020-11-20T13:51:23.642 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-20T13:51:23.887 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 13:51:23 +0000 (0:00:00.043) 0:00:13.832 ******* TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 13:51:23 +0000 (0:00:00.038) 0:00:13.871 ******* TASK [common : Refresh repos] ************************************************** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 13:51:23 +0000 (0:00:00.038) 0:00:13.909 ******* TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 13:51:23 +0000 (0:00:00.040) 0:00:13.950 ******* TASK [common : Refresh repos] ************************************************** skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 13:51:23 +0000 (0:00:00.058) 0:00:14.008 ******* 2020-11-20T13:51:24.524 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-20T13:51:24.525 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.477355", "end": "2020-11-20 13:51:24.500356", "rc": 0, "start": "2020-11-20 13:51:24.023001", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.584396", "end": "2020-11-20 13:51:24.500356", "rc": 0, "start": "2020-11-20 13:51:23.915960", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:51:24.557 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:24 +0000 (0:00:00.700) 0:00:14.709 ******* 2020-11-20T13:51:24.587 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:24 +0000 (0:00:00.029) 0:00:14.739 ******* 2020-11-20T13:51:25.718 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-20T13:51:25.719 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-20T13:51:25.748 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:25 +0000 (0:00:01.161) 0:00:15.900 ******* 2020-11-20T13:51:25.754 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-11-20T13:51:25.784 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:25 +0000 (0:00:00.036) 0:00:15.937 ******* 2020-11-20T13:51:25.789 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-20T13:51:25.791 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:25.823 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:25 +0000 (0:00:00.038) 0:00:15.975 ******* 2020-11-20T13:51:25.827 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:25.858 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:25 +0000 (0:00:00.035) 0:00:16.010 ******* 2020-11-20T13:51:25.863 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-20T13:51:25.864 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:25.893 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:25 +0000 (0:00:00.035) 0:00:16.045 ******* 2020-11-20T13:51:25.897 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:25.928 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:25 +0000 (0:00:00.035) 0:00:16.080 ******* 2020-11-20T13:51:25.933 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-20T13:51:25.935 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:25.966 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:25 +0000 (0:00:00.037) 0:00:16.118 ******* 2020-11-20T13:51:25.970 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:26.006 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:26 +0000 (0:00:00.040) 0:00:16.158 ******* 2020-11-20T13:51:26.012 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-20T13:51:26.014 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:26.062 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:26 +0000 (0:00:00.055) 0:00:16.214 ******* 2020-11-20T13:51:26.068 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:26.102 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:26 +0000 (0:00:00.040) 0:00:16.254 ******* 2020-11-20T13:51:26.491 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-20T13:51:26.493 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-20T13:51:26.524 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:26 +0000 (0:00:00.422) 0:00:16.676 ******* 2020-11-20T13:51:26.540 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-11-20T13:51:26.574 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:26 +0000 (0:00:00.050) 0:00:16.726 ******* 2020-11-20T13:51:29.841 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-20T13:51:29.843 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-20T13:51:29.875 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:29 +0000 (0:00:03.300) 0:00:20.027 ******* 2020-11-20T13:51:29.891 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2020-11-20T13:51:29.925 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:29 +0000 (0:00:00.050) 0:00:20.077 ******* 2020-11-20T13:51:30.407 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T13:51:30.409 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-20T13:51:30.429 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-11-20T13:51:30.865 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T13:51:30.897 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T13:51:31.312 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-20T13:51:31.348 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:31 +0000 (0:00:01.422) 0:00:21.500 ******* 2020-11-20T13:51:31.354 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-11-20T13:51:31.391 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:31 +0000 (0:00:00.042) 0:00:21.543 ******* 2020-11-20T13:51:31.542 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-20T13:51:31.544 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:51:31.578 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:31 +0000 (0:00:00.187) 0:00:21.730 ******* 2020-11-20T13:51:31.584 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:51:31.618 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:31 +0000 (0:00:00.040) 0:00:21.770 ******* 2020-11-20T13:51:32.019 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-20T13:51:32.021 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-20T13:51:32.062 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-11-20T13:51:32.437 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-20T13:51:32.469 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-11-20T13:51:32.844 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-20T13:51:32.879 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:32 +0000 (0:00:01.260) 0:00:23.031 ******* 2020-11-20T13:51:32.885 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-11-20T13:51:32.920 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:32 +0000 (0:00:00.041) 0:00:23.072 ******* 2020-11-20T13:51:32.926 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-20T13:51:32.928 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:32.961 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:32 +0000 (0:00:00.040) 0:00:23.113 ******* 2020-11-20T13:51:32.966 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:33.001 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:32 +0000 (0:00:00.039) 0:00:23.153 ******* 2020-11-20T13:51:33.006 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-20T13:51:33.008 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-20T13:51:33.041 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:33 +0000 (0:00:00.040) 0:00:23.193 ******* 2020-11-20T13:51:33.047 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-11-20T13:51:33.082 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:33 +0000 (0:00:00.040) 0:00:23.234 ******* 2020-11-20T13:51:33.235 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-20T13:51:33.236 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-20T13:51:33.268 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:33 +0000 (0:00:00.185) 0:00:23.420 ******* 2020-11-20T13:51:33.284 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-11-20T13:51:33.317 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:33 +0000 (0:00:00.048) 0:00:23.469 ******* 2020-11-20T13:51:33.323 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-20T13:51:33.325 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:33.336 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:33.368 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:33 +0000 (0:00:00.051) 0:00:23.520 ******* 2020-11-20T13:51:33.373 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:33.385 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:33.386 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:33.418 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:33 +0000 (0:00:00.049) 0:00:23.570 ******* 2020-11-20T13:51:34.303 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-20T13:51:34.305 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-20T13:51:34.337 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:34 +0000 (0:00:00.918) 0:00:24.489 ******* 2020-11-20T13:51:34.363 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-20T13:51:34.364 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-11-20T13:51:34.397 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:34 +0000 (0:00:00.060) 0:00:24.549 ******* 2020-11-20T13:51:34.428 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:34 +0000 (0:00:00.031) 0:00:24.580 ******* 2020-11-20T13:51:34.434 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-20T13:51:34.435 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:34.468 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:34 +0000 (0:00:00.039) 0:00:24.620 ******* 2020-11-20T13:51:34.867 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-20T13:51:34.869 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-20T13:51:34.901 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:34 +0000 (0:00:00.432) 0:00:25.053 ******* 2020-11-20T13:51:34.917 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-11-20T13:51:34.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:34 +0000 (0:00:00.051) 0:00:25.104 ******* 2020-11-20T13:51:35.343 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-20T13:51:35.345 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-20T13:51:35.378 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:35 +0000 (0:00:00.425) 0:00:25.530 ******* 2020-11-20T13:51:35.414 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-20T13:51:35.415 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} TASK [common : Upload nagios check_mem script] ********************************* 2020-11-20T13:51:35.416 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-11-20T13:51:35.448 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:35 +0000 (0:00:00.070) 0:00:25.600 ******* 2020-11-20T13:51:35.479 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:35 +0000 (0:00:00.030) 0:00:25.631 ******* 2020-11-20T13:51:35.485 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-20T13:51:35.486 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:35.488 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-11-20T13:51:35.520 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:35 +0000 (0:00:00.040) 0:00:25.672 ******* 2020-11-20T13:51:35.680 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-20T13:51:35.682 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:35.714 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:35 +0000 (0:00:00.193) 0:00:25.866 ******* 2020-11-20T13:51:35.719 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:51:35.755 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:35 +0000 (0:00:00.041) 0:00:25.907 ******* 2020-11-20T13:51:35.935 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-20T13:51:35.937 INFO:teuthology.task.ansible.out:fatal: [smithi117.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-20T13:51:35.970 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:35 +0000 (0:00:00.214) 0:00:26.122 ******* 2020-11-20T13:51:35.975 INFO:teuthology.task.ansible.out:fatal: [smithi085.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-11-20T13:51:36.010 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:36 +0000 (0:00:00.039) 0:00:26.162 ******* 2020-11-20T13:51:36.015 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-20T13:51:36.017 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:36.051 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:36 +0000 (0:00:00.041) 0:00:26.203 ******* 2020-11-20T13:51:36.056 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:51:36.091 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:36 +0000 (0:00:00.040) 0:00:26.243 ******* 2020-11-20T13:51:36.490 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-20T13:51:36.492 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-20T13:51:36.523 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:36 +0000 (0:00:00.431) 0:00:26.675 ******* 2020-11-20T13:51:36.528 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2020-11-20T13:51:36.561 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:36 +0000 (0:00:00.037) 0:00:26.713 ******* 2020-11-20T13:51:37.206 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-20T13:51:37.207 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:02 UTC", "ActiveEnterTimestampMonotonic": "14747376", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount var-run.mount nss-lookup.target local-fs.target sysinit.target basic.target tmp.mount system.slice network.target time-sync.target systemd-tmpfiles-setup.service systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:02 UTC", "AssertTimestampMonotonic": "14746608", "Before": "xdm.service shutdown.target getty@tty1.service multi-user.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:02 UTC", "ConditionTimestampMonotonic": "14746608", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-11-20T13:51:37.208 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2494", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:02 UTC", "ExecMainStartTimestampMonotonic": "14747340", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:02 UTC", "InactiveExitTimestampMonotonic": "14747376", "InvocationID": "a913639bd2c544e3a579d971fb42f9e6", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitD 2020-11-20T13:51:37.210 INFO:teuthology.task.ansible.out:ATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2494", "MemoryAccounting": "yes", "MemoryCurrent": "1150976", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode 2020-11-20T13:51:37.211 INFO:teuthology.task.ansible.out:": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:02 UTC", "StateChangeTimestampMonotonic": "14747376", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:02 UTC", "WatchdogTimestampMonotonic": "14747375", "WatchdogUSec": "0"}} ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:21 UTC", "ActiveEnterTimestampMonotonic": "15991482", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target system.slice network.target basic.target systemd-tmpfiles-setup.service -.mount systemd-journald.socket tmp.mount local-fs.target sysinit.target var-run.mount time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:21 2020-11-20T13:51:37.213 INFO:teuthology.task.ansible.out:UTC", "AssertTimestampMonotonic": "15990648", "Before": "getty@tty1.service plymouth-quit.service xdm.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:21 UTC", "ConditionTimestampMonotonic": "15990647", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2506", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:21 UTC", "ExecMainStartTimestampMonotonic": "15991445", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv 2020-11-20T13:51:37.215 INFO:teuthology.task.ansible.out:[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:21 UTC", "InactiveExitTimestampMonotonic": "15991482", "InvocationID": "a171d9a6771c4de984c37a567b63529c", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426 2020-11-20T13:51:37.216 INFO:teuthology.task.ansible.out:", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2506", "MemoryAccounting": "yes", "MemoryCurrent": "1245184", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:21 UTC", "StateChangeTimestampMonotonic": "1599 2020-11-20T13:51:37.218 INFO:teuthology.task.ansible.out:1482", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:21 UTC", "WatchdogTimestampMonotonic": "15991481", "WatchdogUSec": "0"}} 2020-11-20T13:51:37.243 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:37 +0000 (0:00:00.681) 0:00:27.395 ******* 2020-11-20T13:51:37.273 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:37 +0000 (0:00:00.030) 0:00:27.425 ******* 2020-11-20T13:51:37.424 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-20T13:51:37.426 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001751", "end": "2020-11-20 13:51:37.407740", "rc": 0, "start": "2020-11-20 13:51:37.405989", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-20T13:51:37.457 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:37 +0000 (0:00:00.184) 0:00:27.609 ******* 2020-11-20T13:51:37.463 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001703", "end": "2020-11-20 13:51:37.438478", "rc": 0, "start": "2020-11-20 13:51:37.436775", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-11-20T13:51:37.497 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:37 +0000 (0:00:00.039) 0:00:27.649 ******* 2020-11-20T13:51:38.354 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-20T13:51:38.356 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-20T13:51:38.386 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:38 +0000 (0:00:00.889) 0:00:28.538 ******* 2020-11-20T13:51:38.402 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-11-20T13:51:38.435 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:38 +0000 (0:00:00.048) 0:00:28.587 ******* 2020-11-20T13:51:39.295 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-20T13:51:39.297 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-20T13:51:39.329 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:39 +0000 (0:00:00.893) 0:00:29.481 ******* 2020-11-20T13:51:39.376 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-11-20T13:51:39.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:39 +0000 (0:00:00.080) 0:00:29.562 ******* 2020-11-20T13:51:40.686 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-20T13:51:40.687 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-20T13:51:40.719 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:40 +0000 (0:00:01.309) 0:00:30.871 ******* 2020-11-20T13:51:40.724 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-11-20T13:51:40.760 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:40 +0000 (0:00:00.041) 0:00:30.912 ******* 2020-11-20T13:51:57.320 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-20T13:51:57.322 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.373039", "end": "2020-11-20 13:51:57.300441", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:51:40.927402", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-20T13:51:57.353 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:16.592) 0:00:47.505 ******* 2020-11-20T13:51:57.359 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.439180", "end": "2020-11-20 13:51:57.323368", "failed_when_result": false, "rc": 0, "start": "2020-11-20 13:51:40.884188", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-11-20T13:51:57.394 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.041) 0:00:47.546 ******* 2020-11-20T13:51:57.786 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-20T13:51:57.788 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-20T13:51:57.820 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.426) 0:00:47.972 ******* 2020-11-20T13:51:57.847 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-11-20T13:51:57.882 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:57 +0000 (0:00:00.061) 0:00:48.034 ******* 2020-11-20T13:51:58.023 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-20T13:51:58.025 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.017499", "end": "2020-11-20 13:51:58.001997", "rc": 0, "start": "2020-11-20 13:51:57.984498", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:51:58.057 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.175) 0:00:48.209 ******* 2020-11-20T13:51:58.085 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.016104", "end": "2020-11-20 13:51:58.063883", "rc": 0, "start": "2020-11-20 13:51:58.047779", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:51:58.119 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.061) 0:00:48.271 ******* 2020-11-20T13:51:58.260 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-20T13:51:58.262 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.010965", "end": "2020-11-20 13:51:58.239337", "rc": 0, "start": "2020-11-20 13:51:58.228372", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:51:58.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.176) 0:00:48.448 ******* 2020-11-20T13:51:58.323 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.008757", "end": "2020-11-20 13:51:58.298888", "rc": 0, "start": "2020-11-20 13:51:58.290131", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:51:58.325 INFO:teuthology.task.ansible.out: 2020-11-20T13:51:58.359 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:51:58 +0000 (0:00:00.063) 0:00:48.511 ******* 2020-11-20T13:52:14.776 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-20T13:52:14.778 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.282988", "end": "2020-11-20 13:52:14.750955", "rc": 0, "start": "2020-11-20 13:51:58.467967", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:14.820 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:14 +0000 (0:00:16.461) 0:01:04.972 ******* 2020-11-20T13:52:14.922 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.367653", "end": "2020-11-20 13:52:14.898945", "rc": 0, "start": "2020-11-20 13:51:58.531292", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:14.958 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:14 +0000 (0:00:00.137) 0:01:05.110 ******* 2020-11-20T13:52:15.015 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-20T13:52:15.017 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-20T13:52:15.051 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.092) 0:01:05.202 ******* 2020-11-20T13:52:15.077 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** 2020-11-20T13:52:15.078 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:15.109 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.058) 0:01:05.261 ******* 2020-11-20T13:52:15.145 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:15.147 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T13:52:15.177 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.067) 0:01:05.329 ******* 2020-11-20T13:52:15.182 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-20T13:52:15.184 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-11-20T13:52:15.216 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.039) 0:01:05.368 ******* 2020-11-20T13:52:15.221 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-20T13:52:15.222 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:15.254 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.038) 0:01:05.406 ******* 2020-11-20T13:52:15.260 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** 2020-11-20T13:52:15.262 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:15.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.041) 0:01:05.448 ******* 2020-11-20T13:52:15.301 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:15.303 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:15.336 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.040) 0:01:05.488 ******* 2020-11-20T13:52:15.342 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:15.343 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T13:52:15.377 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.040) 0:01:05.529 ******* 2020-11-20T13:52:15.383 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T13:52:15.384 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi085.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:52:15.418 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.041) 0:01:05.570 ******* 2020-11-20T13:52:15.423 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-20T13:52:15.425 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:15.458 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.039) 0:01:05.610 ******* 2020-11-20T13:52:15.485 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T13:52:15.487 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:15.523 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.064) 0:01:05.675 ******* 2020-11-20T13:52:15.570 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T13:52:15.572 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi117.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T13:52:15.604 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.080) 0:01:05.756 ******* 2020-11-20T13:52:15.631 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-11-20T13:52:15.632 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:15.670 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.065) 0:01:05.822 ******* 2020-11-20T13:52:15.813 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:15.847 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:15 +0000 (0:00:00.176) 0:01:05.999 ******* 2020-11-20T13:52:16.156 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-20T13:52:16.158 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880335.696376-12676706780370/source", "state": "file", "uid": 0} 2020-11-20T13:52:16.191 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.343) 0:01:06.343 ******* 2020-11-20T13:52:16.217 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-20T13:52:16.219 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-11-20T13:52:16.251 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.060) 0:01:06.403 ******* 2020-11-20T13:52:16.277 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-20T13:52:16.279 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:16.311 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.060) 0:01:06.463 ******* 2020-11-20T13:52:16.349 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-20T13:52:16.350 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-20T13:52:16.352 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605880335.88441-200106338574861/source", "state": "file", "uid": 0} 2020-11-20T13:52:16.383 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.072) 0:01:06.535 ******* 2020-11-20T13:52:16.421 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.037) 0:01:06.573 ******* 2020-11-20T13:52:16.427 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-20T13:52:16.429 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:16.468 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.046) 0:01:06.619 ******* 2020-11-20T13:52:16.473 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-20T13:52:16.475 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-11-20T13:52:16.510 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.042) 0:01:06.662 ******* 2020-11-20T13:52:16.547 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-20T13:52:16.549 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:16.581 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.071) 0:01:06.733 ******* 2020-11-20T13:52:16.618 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-20T13:52:16.620 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2020-11-20T13:52:16.652 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.071) 0:01:06.804 ******* 2020-11-20T13:52:16.658 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-20T13:52:16.660 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:52:16.694 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.041) 0:01:06.846 ******* 2020-11-20T13:52:16.699 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-20T13:52:16.701 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T13:52:16.734 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.039) 0:01:06.886 ******* 2020-11-20T13:52:16.885 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-20T13:52:16.887 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T13:52:16.925 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.190) 0:01:07.077 ******* 2020-11-20T13:52:16.930 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-20T13:52:16.932 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-11-20T13:52:16.965 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:16 +0000 (0:00:00.040) 0:01:07.117 ******* 2020-11-20T13:52:16.971 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-20T13:52:16.973 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:17.006 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.040) 0:01:07.158 ******* 2020-11-20T13:52:17.032 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-20T13:52:17.033 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:17.065 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.059) 0:01:07.217 ******* 2020-11-20T13:52:17.155 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-20T13:52:17.156 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T13:52:17.188 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.123) 0:01:07.340 ******* 2020-11-20T13:52:17.215 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-20T13:52:17.216 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:17.248 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.059) 0:01:07.400 ******* 2020-11-20T13:52:17.274 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-20T13:52:17.275 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:17.307 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.059) 0:01:07.459 ******* 2020-11-20T13:52:17.337 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.029) 0:01:07.489 ******* 2020-11-20T13:52:17.539 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-20T13:52:17.541 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-11-20T13:52:17.574 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.237) 0:01:07.726 ******* 2020-11-20T13:52:17.811 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-20T13:52:17.813 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-20T13:52:17.844 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.269) 0:01:07.996 ******* 2020-11-20T13:52:17.870 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-20T13:52:17.872 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi085"}, "changed": false} 2020-11-20T13:52:17.903 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:17 +0000 (0:00:00.059) 0:01:08.055 ******* 2020-11-20T13:52:18.089 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-20T13:52:18.091 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-11-20T13:52:18.125 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:18 +0000 (0:00:00.221) 0:01:08.276 ******* 2020-11-20T13:52:18.151 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-20T13:52:18.153 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi117"}, "changed": false} 2020-11-20T13:52:18.185 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:18 +0000 (0:00:00.060) 0:01:08.337 ******* 2020-11-20T13:52:18.908 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-20T13:52:18.910 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi117", "ansible_hostname": "smithi117", "ansible_nodename": "smithi117"}, "changed": false, "name": "smithi117"} ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi085", "ansible_hostname": "smithi085", "ansible_nodename": "smithi085"}, "changed": false, "name": "smithi085"} 2020-11-20T13:52:18.942 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:18 +0000 (0:00:00.756) 0:01:09.094 ******* 2020-11-20T13:52:18.973 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:18 +0000 (0:00:00.031) 0:01:09.125 ******* 2020-11-20T13:52:19.373 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-20T13:52:19.374 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-20T13:52:19.406 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:19 +0000 (0:00:00.433) 0:01:09.558 ******* 2020-11-20T13:52:19.412 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-11-20T13:52:19.452 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:19 +0000 (0:00:00.045) 0:01:09.603 ******* 2020-11-20T13:52:19.850 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-20T13:52:19.852 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-20T13:52:19.896 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:19 +0000 (0:00:00.444) 0:01:10.048 ******* 2020-11-20T13:52:19.904 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-11-20T13:52:19.939 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:19 +0000 (0:00:00.043) 0:01:10.091 ******* 2020-11-20T13:52:20.608 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-20T13:52:20.610 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} ok: [smithi085.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-20T13:52:20.642 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:20 +0000 (0:00:00.702) 0:01:10.794 ******* 2020-11-20T13:52:20.672 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:20 +0000 (0:00:00.029) 0:01:10.823 ******* 2020-11-20T13:52:21.091 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-20T13:52:21.093 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-20T13:52:21.126 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.454) 0:01:11.278 ******* 2020-11-20T13:52:21.132 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-11-20T13:52:21.167 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.041) 0:01:11.319 ******* 2020-11-20T13:52:21.318 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T13:52:21.319 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:52:21.353 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.186) 0:01:11.505 ******* 2020-11-20T13:52:21.370 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:52:21.404 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.051) 0:01:11.556 ******* 2020-11-20T13:52:21.544 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T13:52:21.546 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-20T13:52:21.579 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.175) 0:01:11.731 ******* 2020-11-20T13:52:21.596 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-11-20T13:52:21.632 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.052) 0:01:11.784 ******* 2020-11-20T13:52:21.793 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T13:52:21.795 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T13:52:21.831 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.198) 0:01:11.983 ******* 2020-11-20T13:52:21.848 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T13:52:21.887 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:21 +0000 (0:00:00.056) 0:01:12.039 ******* 2020-11-20T13:52:22.286 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T13:52:22.288 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-20T13:52:22.320 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:22 +0000 (0:00:00.433) 0:01:12.472 ******* 2020-11-20T13:52:22.336 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-11-20T13:52:22.370 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:22 +0000 (0:00:00.049) 0:01:12.522 ******* 2020-11-20T13:52:22.770 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T13:52:22.771 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-20T13:52:22.804 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:22 +0000 (0:00:00.434) 0:01:12.956 ******* 2020-11-20T13:52:22.841 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-11-20T13:52:22.878 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:22 +0000 (0:00:00.073) 0:01:13.030 ******* 2020-11-20T13:52:24.609 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T13:52:24.610 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.614337", "end": "2020-11-20 13:52:24.582559", "rc": 0, "start": "2020-11-20 13:52:22.968222", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:24.643 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:24 +0000 (0:00:01.765) 0:01:14.795 ******* 2020-11-20T13:52:24.784 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.721985", "end": "2020-11-20 13:52:24.762178", "rc": 0, "start": "2020-11-20 13:52:23.040193", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:24.820 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:24 +0000 (0:00:00.176) 0:01:14.972 ******* 2020-11-20T13:52:25.117 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T13:52:25.196 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-20T13:52:25.294 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2020-11-20T13:52:25.543 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-20T13:52:25.711 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2020-11-20T13:52:25.960 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-20T13:52:25.996 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:25 +0000 (0:00:01.176) 0:01:16.148 ******* 2020-11-20T13:52:26.126 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2020-11-20T13:52:26.165 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:26 +0000 (0:00:00.168) 0:01:16.316 ******* 2020-11-20T13:52:26.463 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-20T13:52:26.465 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-20T13:52:26.500 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:26 +0000 (0:00:00.335) 0:01:16.652 ******* 2020-11-20T13:52:26.625 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-11-20T13:52:26.661 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:26 +0000 (0:00:00.160) 0:01:16.813 ******* 2020-11-20T13:52:26.959 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** ok: [smithi085.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-20T13:52:27.116 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2020-11-20T13:52:27.387 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-20T13:52:27.547 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2020-11-20T13:52:27.806 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-20T13:52:27.846 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:27 +0000 (0:00:01.185) 0:01:17.998 ******* 2020-11-20T13:52:27.872 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-20T13:52:27.905 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:27 +0000 (0:00:00.059) 0:01:18.057 ******* 2020-11-20T13:52:27.931 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-20T13:52:27.933 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:27.965 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:27 +0000 (0:00:00.060) 0:01:18.117 ******* 2020-11-20T13:52:27.982 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T13:52:27.984 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2020-11-20T13:52:28.020 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.054) 0:01:18.172 ******* 2020-11-20T13:52:28.025 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-20T13:52:28.027 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.060 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.040) 0:01:18.212 ******* 2020-11-20T13:52:28.066 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-20T13:52:28.101 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.040) 0:01:18.253 ******* 2020-11-20T13:52:28.106 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-20T13:52:28.107 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.141 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.040) 0:01:18.293 ******* 2020-11-20T13:52:28.146 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-20T13:52:28.148 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.180 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.039) 0:01:18.332 ******* 2020-11-20T13:52:28.186 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:28.188 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-20T13:52:28.222 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.041) 0:01:18.374 ******* 2020-11-20T13:52:28.228 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-20T13:52:28.229 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.263 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.041) 0:01:18.415 ******* 2020-11-20T13:52:28.269 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-20T13:52:28.270 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.304 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.040) 0:01:18.456 ******* 2020-11-20T13:52:28.310 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-20T13:52:28.311 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.345 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.041) 0:01:18.497 ******* 2020-11-20T13:52:28.350 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-20T13:52:28.352 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.386 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.040) 0:01:18.538 ******* 2020-11-20T13:52:28.392 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:28.394 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2020-11-20T13:52:28.426 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.040) 0:01:18.578 ******* 2020-11-20T13:52:28.463 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-20T13:52:28.464 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.497 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.070) 0:01:18.648 ******* 2020-11-20T13:52:28.523 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-20T13:52:28.524 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.556 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.059) 0:01:18.708 ******* 2020-11-20T13:52:28.583 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-20T13:52:28.583 INFO:teuthology.task.ansible.out:fatal: [smithi085.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006698", "end": "2020-11-20 13:52:28.560866", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:52:28.554168", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-20T13:52:28.619 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.062) 0:01:18.771 ******* 2020-11-20T13:52:28.662 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T13:52:28.664 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.693 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.074) 0:01:18.845 ******* 2020-11-20T13:52:28.761 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-20T13:52:28.763 INFO:teuthology.task.ansible.out:fatal: [smithi117.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007038", "end": "2020-11-20 13:52:28.741112", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:52:28.734074", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-11-20T13:52:28.795 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.101) 0:01:18.947 ******* 2020-11-20T13:52:28.821 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T13:52:28.823 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:28.854 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:28 +0000 (0:00:00.059) 0:01:19.006 ******* 2020-11-20T13:52:30.966 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T13:52:30.967 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-20T13:52:31.004 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:30 +0000 (0:00:02.149) 0:01:21.156 ******* 2020-11-20T13:52:31.041 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-11-20T13:52:31.076 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:31 +0000 (0:00:00.072) 0:01:21.228 ******* 2020-11-20T13:52:32.948 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T13:52:32.949 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-20T13:52:32.982 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:32 +0000 (0:00:01.905) 0:01:23.134 ******* 2020-11-20T13:52:33.020 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-11-20T13:52:33.055 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:33 +0000 (0:00:00.072) 0:01:23.207 ******* 2020-11-20T13:52:35.915 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:52:35.917 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-20T13:52:35.950 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:35 +0000 (0:00:02.894) 0:01:26.102 ******* 2020-11-20T13:52:35.976 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-20T13:52:35.978 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:36.009 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:36 +0000 (0:00:00.059) 0:01:26.161 ******* 2020-11-20T13:52:36.057 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:52:36.058 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-program-options-1.66.0-6.el8.x86_64"]} 2020-11-20T13:52:36.091 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:36 +0000 (0:00:00.082) 0:01:26.243 ******* 2020-11-20T13:52:36.117 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-20T13:52:36.119 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:36.151 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:36 +0000 (0:00:00.059) 0:01:26.303 ******* 2020-11-20T13:52:36.935 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T13:52:36.936 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-20T13:52:36.969 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:36 +0000 (0:00:00.818) 0:01:27.121 ******* 2020-11-20T13:52:36.996 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:52:36.997 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:37.029 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:37 +0000 (0:00:00.060) 0:01:27.181 ******* 2020-11-20T13:52:37.077 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T13:52:37.078 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-11-20T13:52:37.111 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:37 +0000 (0:00:00.081) 0:01:27.263 ******* 2020-11-20T13:52:37.138 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:52:37.139 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:37.171 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:37 +0000 (0:00:00.060) 0:01:27.323 ******* 2020-11-20T13:52:37.333 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-20T13:52:37.334 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.socket dbus.service basic.target sysinit.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ebtables.service ipset.service ip6tables.service shutdown.target iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-20T13:52:37.335 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", 2020-11-20T13:52:37.337 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket sysinit.target system.slice", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-20T13:52:37.339 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-20T13:52:37.369 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:37 +0000 (0:00:00.197) 0:01:27.521 ******* 2020-11-20T13:52:37.473 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target polkit.service sysinit.target dbus.service dbus.socket system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ipset.service iptables.service ebtables.service shutdown.target ip6tables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-11-20T13:52:37.475 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", 2020-11-20T13:52:37.477 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target dbus.socket system.slice", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-11-20T13:52:37.478 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-11-20T13:52:37.508 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:37 +0000 (0:00:00.139) 0:01:27.660 ******* 2020-11-20T13:52:37.654 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-20T13:52:37.656 INFO:teuthology.task.ansible.out:fatal: [smithi117.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-20T13:52:37.687 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:37 +0000 (0:00:00.178) 0:01:27.839 ******* 2020-11-20T13:52:37.820 INFO:teuthology.task.ansible.out:fatal: [smithi085.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-11-20T13:52:37.938 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:37 +0000 (0:00:00.250) 0:01:28.090 ******* 2020-11-20T13:52:38.348 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-20T13:52:38.350 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-11-20T13:52:38.382 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.444) 0:01:28.534 ******* 2020-11-20T13:52:38.412 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.030) 0:01:28.564 ******* 2020-11-20T13:52:38.417 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-20T13:52:38.419 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.452 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.039) 0:01:28.604 ******* 2020-11-20T13:52:38.458 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.492 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.039) 0:01:28.644 ******* 2020-11-20T13:52:38.497 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-20T13:52:38.498 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.531 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.039) 0:01:28.683 ******* 2020-11-20T13:52:38.538 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.573 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.040) 0:01:28.724 ******* 2020-11-20T13:52:38.577 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-20T13:52:38.579 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.611 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.039) 0:01:28.763 ******* 2020-11-20T13:52:38.617 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.652 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.040) 0:01:28.804 ******* 2020-11-20T13:52:38.658 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.692 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.040) 0:01:28.844 ******* 2020-11-20T13:52:38.699 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.735 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.042) 0:01:28.887 ******* 2020-11-20T13:52:38.740 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-20T13:52:38.741 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.774 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.038) 0:01:28.926 ******* 2020-11-20T13:52:38.780 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.812 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.037) 0:01:28.964 ******* 2020-11-20T13:52:38.816 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-20T13:52:38.818 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.848 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.036) 0:01:29.000 ******* 2020-11-20T13:52:38.854 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:38.884 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.036) 0:01:29.036 ******* 2020-11-20T13:52:38.889 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-20T13:52:38.924 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.040) 0:01:29.076 ******* 2020-11-20T13:52:38.963 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.038) 0:01:29.114 ******* 2020-11-20T13:52:38.968 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-20T13:52:38.969 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.001 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:38 +0000 (0:00:00.038) 0:01:29.153 ******* 2020-11-20T13:52:39.007 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.040 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.038) 0:01:29.192 ******* 2020-11-20T13:52:39.046 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-20T13:52:39.048 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.079 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.231 ******* 2020-11-20T13:52:39.086 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.121 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.041) 0:01:29.273 ******* 2020-11-20T13:52:39.125 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-20T13:52:39.127 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.162 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.041) 0:01:29.314 ******* 2020-11-20T13:52:39.167 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.168 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.201 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.353 ******* 2020-11-20T13:52:39.206 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-20T13:52:39.208 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.241 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.393 ******* 2020-11-20T13:52:39.248 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.280 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.432 ******* 2020-11-20T13:52:39.286 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-20T13:52:39.319 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.038) 0:01:29.471 ******* 2020-11-20T13:52:39.358 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.510 ******* 2020-11-20T13:52:39.364 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.396 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.037) 0:01:29.548 ******* 2020-11-20T13:52:39.401 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.434 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.037) 0:01:29.586 ******* 2020-11-20T13:52:39.439 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.470 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.036) 0:01:29.622 ******* 2020-11-20T13:52:39.475 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.509 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.662 ******* 2020-11-20T13:52:39.515 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-20T13:52:39.516 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.549 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.701 ******* 2020-11-20T13:52:39.555 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.588 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.038) 0:01:29.740 ******* 2020-11-20T13:52:39.593 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:52:39.595 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.627 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.038) 0:01:29.779 ******* 2020-11-20T13:52:39.633 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.670 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.042) 0:01:29.822 ******* 2020-11-20T13:52:39.676 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-20T13:52:39.677 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.714 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.044) 0:01:29.866 ******* 2020-11-20T13:52:39.721 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.754 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.906 ******* 2020-11-20T13:52:39.760 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:52:39.761 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.794 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:29.946 ******* 2020-11-20T13:52:39.799 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.834 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.040) 0:01:29.986 ******* 2020-11-20T13:52:39.840 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-20T13:52:39.842 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.875 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.040) 0:01:30.027 ******* 2020-11-20T13:52:39.881 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.914 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:30.066 ******* 2020-11-20T13:52:39.919 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-20T13:52:39.921 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.954 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.040) 0:01:30.106 ******* 2020-11-20T13:52:39.961 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:39.994 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:39 +0000 (0:00:00.039) 0:01:30.146 ******* 2020-11-20T13:52:39.999 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-20T13:52:40.000 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.033 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.185 ******* 2020-11-20T13:52:40.039 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.073 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.225 ******* 2020-11-20T13:52:40.078 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-20T13:52:40.080 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.112 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.038) 0:01:30.264 ******* 2020-11-20T13:52:40.118 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.152 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.304 ******* 2020-11-20T13:52:40.157 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-20T13:52:40.159 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.191 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.038) 0:01:30.343 ******* 2020-11-20T13:52:40.198 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.231 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.383 ******* 2020-11-20T13:52:40.236 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-20T13:52:40.238 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.271 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.423 ******* 2020-11-20T13:52:40.277 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.311 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.040) 0:01:30.463 ******* 2020-11-20T13:52:40.316 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-20T13:52:40.318 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.351 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.040) 0:01:30.503 ******* 2020-11-20T13:52:40.357 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.391 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.040) 0:01:30.543 ******* 2020-11-20T13:52:40.397 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.430 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.038) 0:01:30.581 ******* 2020-11-20T13:52:40.435 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.473 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.043) 0:01:30.625 ******* 2020-11-20T13:52:40.514 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 13:52:40 +0000 (0:00:00.038) 0:01:30.663 ******* 2020-11-20T13:52:40.518 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.554 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.042) 0:01:30.706 ******* 2020-11-20T13:52:40.560 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T13:52:40.562 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.595 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.040) 0:01:30.747 ******* 2020-11-20T13:52:40.601 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.634 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.786 ******* 2020-11-20T13:52:40.639 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T13:52:40.641 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.674 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.040) 0:01:30.826 ******* 2020-11-20T13:52:40.680 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.714 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.866 ******* 2020-11-20T13:52:40.720 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T13:52:40.722 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.753 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.905 ******* 2020-11-20T13:52:40.759 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.793 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.944 ******* 2020-11-20T13:52:40.798 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T13:52:40.800 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.832 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:30.984 ******* 2020-11-20T13:52:40.838 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.873 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.041) 0:01:31.025 ******* 2020-11-20T13:52:40.879 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T13:52:40.881 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.914 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.040) 0:01:31.066 ******* 2020-11-20T13:52:40.919 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.953 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:31.105 ******* 2020-11-20T13:52:40.958 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T13:52:40.960 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:40.993 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:40 +0000 (0:00:00.039) 0:01:31.145 ******* 2020-11-20T13:52:40.998 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.032 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.039) 0:01:31.184 ******* 2020-11-20T13:52:41.038 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T13:52:41.039 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.070 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.038) 0:01:31.222 ******* 2020-11-20T13:52:41.075 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.109 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.039) 0:01:31.261 ******* 2020-11-20T13:52:41.114 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:41.116 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.149 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.040) 0:01:31.301 ******* 2020-11-20T13:52:41.156 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.190 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.040) 0:01:31.342 ******* 2020-11-20T13:52:41.195 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T13:52:41.196 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.231 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.041) 0:01:31.383 ******* 2020-11-20T13:52:41.236 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.269 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.038) 0:01:31.421 ******* 2020-11-20T13:52:41.274 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T13:52:41.275 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.309 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.039) 0:01:31.461 ******* 2020-11-20T13:52:41.315 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.348 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.039) 0:01:31.500 ******* 2020-11-20T13:52:41.353 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T13:52:41.355 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.389 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.040) 0:01:31.541 ******* 2020-11-20T13:52:41.395 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.430 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.040) 0:01:31.582 ******* 2020-11-20T13:52:41.435 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T13:52:41.437 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.471 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.041) 0:01:31.623 ******* 2020-11-20T13:52:41.476 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:41.511 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.040) 0:01:31.663 ******* 2020-11-20T13:52:41.672 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:52:41.674 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-20T13:52:41.707 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.196) 0:01:31.859 ******* 2020-11-20T13:52:41.724 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2020-11-20T13:52:41.759 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.051) 0:01:31.911 ******* 2020-11-20T13:52:41.909 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:41.944 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.185) 0:01:32.096 ******* 2020-11-20T13:52:41.962 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:52:41.997 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:41 +0000 (0:00:00.053) 0:01:32.149 ******* 2020-11-20T13:52:42.139 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:52:42.141 INFO:teuthology.task.ansible.out:[WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-20T13:52:42.174 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.177) 0:01:32.326 ******* 2020-11-20T13:52:42.192 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2020-11-20T13:52:42.225 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.050) 0:01:32.376 ******* 2020-11-20T13:52:42.230 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T13:52:42.231 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.263 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.038) 0:01:32.415 ******* 2020-11-20T13:52:42.269 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.302 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.454 ******* 2020-11-20T13:52:42.308 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T13:52:42.309 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.344 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.042) 0:01:32.496 ******* 2020-11-20T13:52:42.351 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.384 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.536 ******* 2020-11-20T13:52:42.389 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T13:52:42.391 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.424 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.576 ******* 2020-11-20T13:52:42.430 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.464 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.616 ******* 2020-11-20T13:52:42.469 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:52:42.470 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.503 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.655 ******* 2020-11-20T13:52:42.509 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.544 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.695 ******* 2020-11-20T13:52:42.548 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T13:52:42.550 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.583 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.040) 0:01:32.735 ******* 2020-11-20T13:52:42.589 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.623 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.775 ******* 2020-11-20T13:52:42.629 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T13:52:42.630 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.664 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.040) 0:01:32.816 ******* 2020-11-20T13:52:42.670 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.703 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.855 ******* 2020-11-20T13:52:42.708 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T13:52:42.710 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.747 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.044) 0:01:32.899 ******* 2020-11-20T13:52:42.754 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.787 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.039) 0:01:32.939 ******* 2020-11-20T13:52:42.793 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T13:52:42.795 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.830 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.042) 0:01:32.982 ******* 2020-11-20T13:52:42.836 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.872 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.041) 0:01:33.024 ******* 2020-11-20T13:52:42.878 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.913 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.041) 0:01:33.065 ******* 2020-11-20T13:52:42.919 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.955 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.041) 0:01:33.107 ******* 2020-11-20T13:52:42.960 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-20T13:52:42.962 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:42.996 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:42 +0000 (0:00:00.040) 0:01:33.148 ******* 2020-11-20T13:52:43.002 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.038 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.041) 0:01:33.190 ******* 2020-11-20T13:52:43.044 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-20T13:52:43.046 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.080 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.041) 0:01:33.232 ******* 2020-11-20T13:52:43.086 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.130 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.050) 0:01:33.282 ******* 2020-11-20T13:52:43.138 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T13:52:43.139 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.178 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.047) 0:01:33.330 ******* 2020-11-20T13:52:43.184 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.220 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.041) 0:01:33.372 ******* 2020-11-20T13:52:43.226 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T13:52:43.228 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.276 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.055) 0:01:33.428 ******* 2020-11-20T13:52:43.284 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.319 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.043) 0:01:33.471 ******* 2020-11-20T13:52:43.326 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-20T13:52:43.328 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.362 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.042) 0:01:33.514 ******* 2020-11-20T13:52:43.367 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.401 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.038) 0:01:33.553 ******* 2020-11-20T13:52:43.406 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-20T13:52:43.407 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.441 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.039) 0:01:33.593 ******* 2020-11-20T13:52:43.447 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.480 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.038) 0:01:33.632 ******* 2020-11-20T13:52:43.484 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-20T13:52:43.486 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.519 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.038) 0:01:33.671 ******* 2020-11-20T13:52:43.524 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.557 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.038) 0:01:33.709 ******* 2020-11-20T13:52:43.562 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-20T13:52:43.563 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.595 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.038) 0:01:33.747 ******* 2020-11-20T13:52:43.601 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.635 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.039) 0:01:33.787 ******* 2020-11-20T13:52:43.640 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-20T13:52:43.642 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.674 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.039) 0:01:33.826 ******* 2020-11-20T13:52:43.680 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.718 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.043) 0:01:33.870 ******* 2020-11-20T13:52:43.723 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-20T13:52:43.725 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.759 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.040) 0:01:33.911 ******* 2020-11-20T13:52:43.764 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.799 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.040) 0:01:33.951 ******* 2020-11-20T13:52:43.804 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-20T13:52:43.806 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.839 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.040) 0:01:33.991 ******* 2020-11-20T13:52:43.845 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.879 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.039) 0:01:34.031 ******* 2020-11-20T13:52:43.885 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-20T13:52:43.886 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.922 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.043) 0:01:34.074 ******* 2020-11-20T13:52:43.930 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:43.968 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:43 +0000 (0:00:00.045) 0:01:34.120 ******* 2020-11-20T13:52:43.973 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-20T13:52:43.975 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.008 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.039) 0:01:34.160 ******* 2020-11-20T13:52:44.014 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.048 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.040) 0:01:34.200 ******* 2020-11-20T13:52:44.053 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:44.055 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T13:52:44.098 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.050) 0:01:34.250 ******* 2020-11-20T13:52:44.104 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T13:52:44.139 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.041) 0:01:34.291 ******* 2020-11-20T13:52:44.147 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-20T13:52:44.148 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.149 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T13:52:44.150 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.162 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.173 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.184 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.184 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.186 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.188 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T13:52:44.246 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.106) 0:01:34.398 ******* 2020-11-20T13:52:44.253 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.254 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.256 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.257 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.303 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.056) 0:01:34.455 ******* 2020-11-20T13:52:44.434 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-20T13:52:44.436 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.002813", "end": "2020-11-20 13:52:44.412458", "rc": 0, "start": "2020-11-20 13:52:44.409645", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T13:52:44.480 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.176) 0:01:34.632 ******* 2020-11-20T13:52:44.506 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-20T13:52:44.508 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.545 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.065) 0:01:34.697 ******* 2020-11-20T13:52:44.551 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-20T13:52:44.553 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003057", "end": "2020-11-20 13:52:44.491687", "rc": 0, "start": "2020-11-20 13:52:44.488630", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T13:52:44.594 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.048) 0:01:34.746 ******* 2020-11-20T13:52:44.599 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-20T13:52:44.601 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.635 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.041) 0:01:34.787 ******* 2020-11-20T13:52:44.641 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-20T13:52:44.643 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.681 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.045) 0:01:34.833 ******* 2020-11-20T13:52:44.686 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-20T13:52:44.688 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.722 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.040) 0:01:34.873 ******* 2020-11-20T13:52:44.727 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-20T13:52:44.728 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.761 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.039) 0:01:34.913 ******* 2020-11-20T13:52:44.766 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-20T13:52:44.768 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.804 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.042) 0:01:34.956 ******* 2020-11-20T13:52:44.810 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.844 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.040) 0:01:34.996 ******* 2020-11-20T13:52:44.871 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-20T13:52:44.873 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:44.904 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:44 +0000 (0:00:00.059) 0:01:35.056 ******* 2020-11-20T13:52:45.734 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-20T13:52:45.735 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-20T13:52:45.770 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:45 +0000 (0:00:00.866) 0:01:35.922 ******* 2020-11-20T13:52:45.809 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:52:45.811 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529271808, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179998, 'block_used': 12915110, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179998, "block_size": 4096, "block_total": 240095108, "block_used": 12915110, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529271808, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-20T13:52:45.845 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:45 +0000 (0:00:00.074) 0:01:35.997 ******* 2020-11-20T13:52:45.853 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-20T13:52:45.855 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-11-20T13:52:45.888 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:45 +0000 (0:00:00.043) 0:01:36.040 ******* 2020-11-20T13:52:45.926 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:52:45.928 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529275904, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179999, 'block_used': 12915109, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179999, "block_size": 4096, "block_total": 240095108, "block_used": 12915109, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529275904, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}} 2020-11-20T13:52:45.963 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:45 +0000 (0:00:00.074) 0:01:36.115 ******* 2020-11-20T13:52:46.031 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-20T13:52:46.033 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005030", "end": "2020-11-20 13:52:46.013204", "rc": 0, "start": "2020-11-20 13:52:46.008174", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:46.076 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:46 +0000 (0:00:00.112) 0:01:36.227 ******* 2020-11-20T13:52:46.115 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-20T13:52:46.117 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529271808, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179998, 'block_used': 12915110, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179998, "block_size": 4096, "block_total": 240095108, "block_used": 12915110, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529271808, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:46.154 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:46 +0000 (0:00:00.078) 0:01:36.306 ******* 2020-11-20T13:52:46.161 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.004970", "end": "2020-11-20 13:52:46.134947", "rc": 0, "start": "2020-11-20 13:52:46.129977", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:46.195 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:46 +0000 (0:00:00.041) 0:01:36.347 ******* 2020-11-20T13:52:46.202 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T13:52:46.240 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:46 +0000 (0:00:00.044) 0:01:36.392 ******* 2020-11-20T13:52:46.247 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-20T13:52:46.249 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529275904, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179999, 'block_used': 12915109, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179999, "block_size": 4096, "block_total": 240095108, "block_used": 12915109, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529275904, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:46.283 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:46 +0000 (0:00:00.042) 0:01:36.435 ******* 2020-11-20T13:52:46.310 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T13:52:46.345 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:46 +0000 (0:00:00.061) 0:01:36.497 ******* 2020-11-20T13:52:47.725 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T13:52:47.727 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt6KYqy6KdQY0ToMZu3h2cHucLrVLcI5gi'], 'uuids': ['ee0dc69d-92c7-4bd7-be65-7558a2effaf3'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.266102", "end": "2020-11-20 13:52:47.691982", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt6KYqy6KdQY0ToMZu3h2cHucLrVLcI5gi"], "labels": [], "masters": [], "uuids": ["ee0dc69d-92c7-4bd7-be65-7558a2effaf3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:46.425880", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:47.996 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDfEXRrDe6V4PpHCQLNus8AEFi5NmGqsA0'], 'uuids': ['c4109bb0-f4c8-44f3-8183-86aa373f2059'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.433258", "end": "2020-11-20 13:52:47.965821", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDfEXRrDe6V4PpHCQLNus8AEFi5NmGqsA0"], "labels": [], "masters": [], "uuids": ["c4109bb0-f4c8-44f3-8183-86aa373f2059"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:46.532563", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:48.983 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-BBCMQp-xXrk-4GSd-esgS-5A9M-vSnu-meZVQH', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400X7400BGN', 'nvme-nvme.8086-50484654363230343030583734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.076646", "end": "2020-11-20 13:52:48.957792", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-BBCMQp-xXrk-4GSd-esgS-5A9M-vSnu-meZVQH", "nvme-INTEL_SSDPEDMD400G4_PHFT620400X7400BGN", "nvme-nvme.8086-50484654363230343030583734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:47.881146", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T13:52:48.985 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:49.246 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-EX9QcR-SHTt-uMI4-kfUU-Fbdt-j4Uw-h1YaH9', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400V6400BGN', 'nvme-nvme.8086-50484654363230343030563634303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.065075", "end": "2020-11-20 13:52:49.218765", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-EX9QcR-SHTt-uMI4-kfUU-Fbdt-j4Uw-h1YaH9", "nvme-INTEL_SSDPEDMD400G4_PHFT620400V6400BGN", "nvme-nvme.8086-50484654363230343030563634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:48.153690", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T13:52:49.249 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:50.243 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt5yTGuemExAyfQbaC838VqTgia04DZGWd'], 'uuids': ['ead48764-84de-49c3-a07a-89648e7360e6'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.056425", "end": "2020-11-20 13:52:50.209364", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt5yTGuemExAyfQbaC838VqTgia04DZGWd"], "labels": [], "masters": [], "uuids": ["ead48764-84de-49c3-a07a-89648e7360e6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:49.152939", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:50.492 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDkaNjzVAlWK8oR08qLdmoGffLkdaVEnoY'], 'uuids': ['af6c7ae3-b52d-49b8-90d4-1de87416d06d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.053304", "end": "2020-11-20 13:52:50.463185", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDkaNjzVAlWK8oR08qLdmoGffLkdaVEnoY"], "labels": [], "masters": [], "uuids": ["af6c7ae3-b52d-49b8-90d4-1de87416d06d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:49.409881", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:51.498 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltmIT3SjqPY0QIqe7LrGD53TZjUUofiNNM'], 'uuids': ['f9024a92-ebf0-454d-bc1b-8455ad0206c5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.071455", "end": "2020-11-20 13:52:51.466634", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltmIT3SjqPY0QIqe7LrGD53TZjUUofiNNM"], "labels": [], "masters": [], "uuids": ["f9024a92-ebf0-454d-bc1b-8455ad0206c5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:50.395179", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:51.738 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDpHIdE1RLwkdnbTuzYcN8GbBzjG9scKl9'], 'uuids': ['5f282c5a-0f83-4fea-ad67-a55727a00c59'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.052848", "end": "2020-11-20 13:52:51.704464", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDpHIdE1RLwkdnbTuzYcN8GbBzjG9scKl9"], "labels": [], "masters": [], "uuids": ["5f282c5a-0f83-4fea-ad67-a55727a00c59"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:50.651616", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:52.747 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltTf9kYu0va59hLFpVNP3lUPnSSgGRy6JV'], 'uuids': ['84ce0898-f4e1-4751-ad50-db996a9610e5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.062962", "end": "2020-11-20 13:52:52.716397", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltTf9kYu0va59hLFpVNP3lUPnSSgGRy6JV"], "labels": [], "masters": [], "uuids": ["84ce0898-f4e1-4751-ad50-db996a9610e5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:51.653435", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:52.759 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QA6H', 'wwn-0x5000c50091e09aef'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QA6H', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e09aef', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QA6H-part1', 'wwn-0x5000c50091e09aef-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QA6H", "wwn-0x5000c50091e09aef"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QA6H-part1", "wwn-0x5000c50091e09aef-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QA6H", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e09aef"}}, 2020-11-20T13:52:52.760 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T13:52:52.977 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMD1mTK7IzSlacUcbBujNRsN3AyqWLXsIRn'], 'uuids': ['b68b571b-b5de-4c18-b550-ac6c712a3958'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.055258", "end": "2020-11-20 13:52:52.946095", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMD1mTK7IzSlacUcbBujNRsN3AyqWLXsIRn"], "labels": [], "masters": [], "uuids": ["b68b571b-b5de-4c18-b550-ac6c712a3958"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:51.890837", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:52.989 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R17F', 'wwn-0x5000c50091e2fc17'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5R17F', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e2fc17', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R17F-part1', 'wwn-0x5000c50091e2fc17-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R17F", "wwn-0x5000c50091e2fc17"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R17F-part1", "wwn-0x5000c50091e2fc17-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5R17F", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e2fc17"}}, 2020-11-20T13:52:52.990 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T13:52:53.994 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltN2JFLn4tkV0xCT4eFOdBTCDtcG2D3b5o'], 'uuids': ['11969932-ed8e-4f88-8439-b4d41bbe19e5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.056887", "end": "2020-11-20 13:52:53.966035", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltN2JFLn4tkV0xCT4eFOdBTCDtcG2D3b5o"], "labels": [], "masters": [], "uuids": ["11969932-ed8e-4f88-8439-b4d41bbe19e5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:52.909148", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:54.031 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:54 +0000 (0:00:07.686) 0:01:44.183 ******* 2020-11-20T13:52:54.267 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDHF6dwfz1OdchIGT4IjTdxczAJ2fo9dSZ'], 'uuids': ['f06c5c4d-2753-4e42-9288-82c12d785b85'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.098927", "end": "2020-11-20 13:52:54.238442", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDHF6dwfz1OdchIGT4IjTdxczAJ2fo9dSZ"], "labels": [], "masters": [], "uuids": ["f06c5c4d-2753-4e42-9288-82c12d785b85"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:53.139515", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T13:52:54.310 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:54 +0000 (0:00:00.277) 0:01:44.461 ******* 2020-11-20T13:52:54.390 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T13:52:54.391 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt6KYqy6KdQY0ToMZu3h2cHucLrVLcI5gi'], 'uuids': ['ee0dc69d-92c7-4bd7-be65-7558a2effaf3'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.140175", "end": "2020-11-20 13:52:54.367265", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt6KYqy6KdQY0ToMZu3h2cHucLrVLcI5gi"], "labels": [], "masters": [], "uuids": ["ee0dc69d-92c7-4bd7-be65-7558a2effaf3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:54.227090", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137588 s, 838 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137588 s, 838 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:54.609 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-BBCMQp-xXrk-4GSd-esgS-5A9M-vSnu-meZVQH', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400X7400BGN', 'nvme-nvme.8086-50484654363230343030583734303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037166", "end": "2020-11-20 13:52:54.583358", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-BBCMQp-xXrk-4GSd-esgS-5A9M-vSnu-meZVQH", "nvme-INTEL_SSDPEDMD400G4_PHFT620400X7400BGN", "nvme-nvme.8086-50484654363230343030583734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:54.546192", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0345869 s, 3.3 GB/s", "std 2020-11-20T13:52:54.611 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0345869 s, 3.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:54.664 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDfEXRrDe6V4PpHCQLNus8AEFi5NmGqsA0'], 'uuids': ['c4109bb0-f4c8-44f3-8183-86aa373f2059'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.137937", "end": "2020-11-20 13:52:54.637074", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDfEXRrDe6V4PpHCQLNus8AEFi5NmGqsA0"], "labels": [], "masters": [], "uuids": ["c4109bb0-f4c8-44f3-8183-86aa373f2059"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:54.499137", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.135394 s, 852 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.135394 s, 852 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:54.875 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-EX9QcR-SHTt-uMI4-kfUU-Fbdt-j4Uw-h1YaH9', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400V6400BGN', 'nvme-nvme.8086-50484654363230343030563634303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.035755", "end": "2020-11-20 13:52:54.852891", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-EX9QcR-SHTt-uMI4-kfUU-Fbdt-j4Uw-h1YaH9", "nvme-INTEL_SSDPEDMD400G4_PHFT620400V6400BGN", "nvme-nvme.8086-50484654363230343030563634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:54.817136", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0333746 s, 3.5 GB/s", "std 2020-11-20T13:52:54.876 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0333746 s, 3.5 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:54.938 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt5yTGuemExAyfQbaC838VqTgia04DZGWd'], 'uuids': ['ead48764-84de-49c3-a07a-89648e7360e6'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.142279", "end": "2020-11-20 13:52:54.914166", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1lt5yTGuemExAyfQbaC838VqTgia04DZGWd"], "labels": [], "masters": [], "uuids": ["ead48764-84de-49c3-a07a-89648e7360e6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:54.771887", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139667 s, 826 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139667 s, 826 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:55.219 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDkaNjzVAlWK8oR08qLdmoGffLkdaVEnoY'], 'uuids': ['af6c7ae3-b52d-49b8-90d4-1de87416d06d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.153674", "end": "2020-11-20 13:52:55.197081", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDkaNjzVAlWK8oR08qLdmoGffLkdaVEnoY"], "labels": [], "masters": [], "uuids": ["af6c7ae3-b52d-49b8-90d4-1de87416d06d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:55.043407", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.151064 s, 764 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.151064 s, 764 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:55.251 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltmIT3SjqPY0QIqe7LrGD53TZjUUofiNNM'], 'uuids': ['f9024a92-ebf0-454d-bc1b-8455ad0206c5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.140313", "end": "2020-11-20 13:52:55.230871", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltmIT3SjqPY0QIqe7LrGD53TZjUUofiNNM"], "labels": [], "masters": [], "uuids": ["f9024a92-ebf0-454d-bc1b-8455ad0206c5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:55.090558", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137783 s, 837 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137783 s, 837 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:55.534 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDpHIdE1RLwkdnbTuzYcN8GbBzjG9scKl9'], 'uuids': ['5f282c5a-0f83-4fea-ad67-a55727a00c59'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.140259", "end": "2020-11-20 13:52:55.513540", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDpHIdE1RLwkdnbTuzYcN8GbBzjG9scKl9"], "labels": [], "masters": [], "uuids": ["5f282c5a-0f83-4fea-ad67-a55727a00c59"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:55.373281", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137714 s, 838 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137714 s, 838 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:55.587 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltTf9kYu0va59hLFpVNP3lUPnSSgGRy6JV'], 'uuids': ['84ce0898-f4e1-4751-ad50-db996a9610e5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.154114", "end": "2020-11-20 13:52:55.561698", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltTf9kYu0va59hLFpVNP3lUPnSSgGRy6JV"], "labels": [], "masters": [], "uuids": ["84ce0898-f4e1-4751-ad50-db996a9610e5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:55.407584", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15161 s, 761 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15161 s, 761 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:55.599 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QA6H', 'wwn-0x5000c50091e09aef'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QA6H', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e09aef', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QA6H-part1', 'wwn-0x5000c50091e09aef-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QA6H", "wwn-0x5000c50091e09aef"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QA6H-part1", "wwn-0x5000c50091e09aef-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QA6H", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e09aef"}}, 2020-11-20T13:52:55.601 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T13:52:55.850 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMD1mTK7IzSlacUcbBujNRsN3AyqWLXsIRn'], 'uuids': ['b68b571b-b5de-4c18-b550-ac6c712a3958'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.140182", "end": "2020-11-20 13:52:55.830272", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMD1mTK7IzSlacUcbBujNRsN3AyqWLXsIRn"], "labels": [], "masters": [], "uuids": ["b68b571b-b5de-4c18-b550-ac6c712a3958"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:55.690090", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137742 s, 837 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137742 s, 837 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:55.964 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R17F', 'wwn-0x5000c50091e2fc17'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5R17F', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e2fc17', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5R17F-part1', 'wwn-0x5000c50091e2fc17-part1'], 'uuids': ['9bdf28e7-5b1a-4776-884c-191bd8ad39f5'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R17F", "wwn-0x5000c50091e2fc17"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R17F-part1", "wwn-0x5000c50091e2fc17-part1"], "labels": [], "masters": [], "uuids": ["9bdf28e7-5b1a-4776-884c-191bd8ad39f5"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5R17F", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e2fc17"}}, 2020-11-20T13:52:55.966 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltN2JFLn4tkV0xCT4eFOdBTCDtcG2D3b5o'], 'uuids': ['11969932-ed8e-4f88-8439-b4d41bbe19e5'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.142532", "end": "2020-11-20 13:52:55.887775", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-TQh3igMLsKsUAYA7mVkoxDsdLbr9T1ltN2JFLn4tkV0xCT4eFOdBTCDtcG2D3b5o"], "labels": [], "masters": [], "uuids": ["11969932-ed8e-4f88-8439-b4d41bbe19e5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:55.745243", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139943 s, 824 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139943 s, 824 MB/s"], "stdout": "", "stdout_lines": []} Friday 20 November 2020 13:52:55 +0000 (0:00:01.644) 0:01:46.106 ******* 2020-11-20T13:52:56.172 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDHF6dwfz1OdchIGT4IjTdxczAJ2fo9dSZ'], 'uuids': ['f06c5c4d-2753-4e42-9288-82c12d785b85'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.140073", "end": "2020-11-20 13:52:56.151551", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-y78eR5cOXtI8E1K0aK1V6XldHWrg2aMDHF6dwfz1OdchIGT4IjTdxczAJ2fo9dSZ"], "labels": [], "masters": [], "uuids": ["f06c5c4d-2753-4e42-9288-82c12d785b85"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 13:52:56.011478", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.137666 s, 838 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.137666 s, 838 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:56.224 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.270) 0:01:46.376 ******* 2020-11-20T13:52:56.232 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-20T13:52:56.233 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.079961", "end": "2020-11-20 13:52:56.197612", "rc": 0, "start": "2020-11-20 13:52:56.117651", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:56.269 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.044) 0:01:46.421 ******* 2020-11-20T13:52:56.495 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.078361", "end": "2020-11-20 13:52:56.465228", "rc": 0, "start": "2020-11-20 13:52:56.386867", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:56.529 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.260) 0:01:46.681 ******* 2020-11-20T13:52:56.536 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-20T13:52:56.537 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.047425", "end": "2020-11-20 13:52:56.480464", "rc": 0, "start": "2020-11-20 13:52:56.433039", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:56.571 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.041) 0:01:46.723 ******* 2020-11-20T13:52:56.597 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-20T13:52:56.634 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.062) 0:01:46.786 ******* 2020-11-20T13:52:56.660 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-20T13:52:56.662 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:56.698 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.064) 0:01:46.850 ******* 2020-11-20T13:52:56.725 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-20T13:52:56.727 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:56.759 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.061) 0:01:46.911 ******* 2020-11-20T13:52:56.767 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-20T13:52:56.769 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.048535", "end": "2020-11-20 13:52:56.738844", "rc": 0, "start": "2020-11-20 13:52:56.690309", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:52:56.802 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.042) 0:01:46.954 ******* 2020-11-20T13:52:56.807 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-20T13:52:56.809 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:56.844 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.042) 0:01:46.996 ******* 2020-11-20T13:52:56.850 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-20T13:52:56.887 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.043) 0:01:47.039 ******* 2020-11-20T13:52:56.893 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-20T13:52:56.894 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:56.928 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.041) 0:01:47.080 ******* 2020-11-20T13:52:56.934 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-20T13:52:56.935 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:56.973 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:56 +0000 (0:00:00.044) 0:01:47.125 ******* 2020-11-20T13:52:56.999 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-20T13:52:57.001 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.033 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.060) 0:01:47.185 ******* 2020-11-20T13:52:57.060 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-20T13:52:57.062 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.096 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.062) 0:01:47.248 ******* 2020-11-20T13:52:57.122 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-20T13:52:57.124 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.156 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.060) 0:01:47.308 ******* 2020-11-20T13:52:57.186 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.029) 0:01:47.338 ******* 2020-11-20T13:52:57.191 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:52:57.193 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529271808, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179998, 'block_used': 12915110, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179998, "block_size": 4096, "block_total": 240095108, "block_used": 12915110, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529271808, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.229 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.042) 0:01:47.380 ******* 2020-11-20T13:52:57.255 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-20T13:52:57.257 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.288 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.059) 0:01:47.440 ******* 2020-11-20T13:52:57.321 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:57.323 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.354 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.065) 0:01:47.505 ******* 2020-11-20T13:52:57.370 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-20T13:52:57.371 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T13:52:57.404 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.050) 0:01:47.556 ******* 2020-11-20T13:52:57.409 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:57.411 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.447 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.042) 0:01:47.599 ******* 2020-11-20T13:52:57.452 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T13:52:57.454 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930529275904, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227179999, 'block_used': 12915109, 'inode_total': 61054976, 'inode_available': 60990124, 'inode_used': 64852, 'uuid': '9bdf28e7-5b1a-4776-884c-191bd8ad39f5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227179999, "block_size": 4096, "block_total": 240095108, "block_used": 12915109, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60990124, "inode_total": 61054976, "inode_used": 64852, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930529275904, "size_total": 983429562368, "uuid": "9bdf28e7-5b1a-4776-884c-191bd8ad39f5"}, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.488 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.041) 0:01:47.640 ******* 2020-11-20T13:52:57.493 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-20T13:52:57.495 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.529 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.041) 0:01:47.681 ******* 2020-11-20T13:52:57.535 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-20T13:52:57.537 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.570 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.040) 0:01:47.722 ******* 2020-11-20T13:52:57.576 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.611 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.041) 0:01:47.763 ******* 2020-11-20T13:52:57.617 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:57.619 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.652 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.040) 0:01:47.804 ******* 2020-11-20T13:52:57.659 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.693 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.041) 0:01:47.845 ******* 2020-11-20T13:52:57.700 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:52:57.702 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.735 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.041) 0:01:47.887 ******* 2020-11-20T13:52:57.762 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-20T13:52:57.763 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.796 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.060) 0:01:47.948 ******* 2020-11-20T13:52:57.822 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-20T13:52:57.824 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.856 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.060) 0:01:48.008 ******* 2020-11-20T13:52:57.882 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-20T13:52:57.884 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:52:57.917 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:57 +0000 (0:00:00.060) 0:01:48.069 ******* 2020-11-20T13:52:58.286 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-20T13:52:58.288 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-20T13:52:58.321 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:58 +0000 (0:00:00.404) 0:01:48.473 ******* 2020-11-20T13:52:58.358 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-20T13:52:58.394 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:52:58 +0000 (0:00:00.072) 0:01:48.546 ******* 2020-11-20T13:52:58.977 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T13:52:58.979 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:52:59.294 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:52:59.306 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:52:59.619 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:52:59.651 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:52:59.954 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:52:59.997 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:00.270 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:00.304 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:01.910) 0:01:50.456 ******* 2020-11-20T13:53:00.352 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T13:53:00.391 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.087) 0:01:50.543 ******* 2020-11-20T13:53:00.491 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-20T13:53:00.493 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T13:53:00.525 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.134) 0:01:50.677 ******* 2020-11-20T13:53:00.583 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T13:53:00.620 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:00 +0000 (0:00:00.094) 0:01:50.772 ******* 2020-11-20T13:53:00.733 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T13:53:00.735 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:00.829 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:00.904 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:01.011 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:01.085 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:01.191 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:01.266 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:01.277 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:01.313 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:01 +0000 (0:00:00.693) 0:01:51.465 ******* 2020-11-20T13:53:01.372 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T13:53:01.383 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:01.419 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:01 +0000 (0:00:00.106) 0:01:51.571 ******* 2020-11-20T13:53:01.513 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:53:01.546 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:01 +0000 (0:00:00.127) 0:01:51.698 ******* 2020-11-20T13:53:01.583 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-20T13:53:01.585 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-20T13:53:01.620 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:01 +0000 (0:00:00.073) 0:01:51.772 ******* 2020-11-20T13:53:01.626 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-20T13:53:01.628 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T13:53:01.662 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:01 +0000 (0:00:00.041) 0:01:51.814 ******* 2020-11-20T13:53:01.698 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-20T13:53:01.700 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-20T13:53:01.732 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:01 +0000 (0:00:00.070) 0:01:51.884 ******* 2020-11-20T13:53:03.213 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-20T13:53:03.215 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true} changed: [smithi085.front.sepia.ceph.com] => {"changed": true} 2020-11-20T13:53:03.249 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:01.516) 0:01:53.401 ******* 2020-11-20T13:53:03.279 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.030) 0:01:53.431 ******* 2020-11-20T13:53:03.807 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-11-20T13:53:03.809 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-20T13:53:03.840 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.561) 0:01:53.992 ******* 2020-11-20T13:53:03.873 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.032) 0:01:54.025 ******* 2020-11-20T13:53:03.878 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:03.912 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.039) 0:01:54.064 ******* 2020-11-20T13:53:03.918 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:03.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.040) 0:01:54.104 ******* 2020-11-20T13:53:03.959 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-20T13:53:03.960 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:03.993 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:03 +0000 (0:00:00.040) 0:01:54.145 ******* 2020-11-20T13:53:03.999 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.034 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.186 ******* 2020-11-20T13:53:04.040 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-20T13:53:04.041 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.075 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.227 ******* 2020-11-20T13:53:04.081 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.113 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.038) 0:01:54.265 ******* 2020-11-20T13:53:04.118 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-20T13:53:04.120 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.152 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.039) 0:01:54.304 ******* 2020-11-20T13:53:04.159 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.192 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.344 ******* 2020-11-20T13:53:04.198 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-20T13:53:04.200 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.233 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.385 ******* 2020-11-20T13:53:04.239 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.273 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.039) 0:01:54.425 ******* 2020-11-20T13:53:04.278 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-20T13:53:04.280 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.312 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.039) 0:01:54.464 ******* 2020-11-20T13:53:04.318 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.352 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.504 ******* 2020-11-20T13:53:04.358 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-20T13:53:04.360 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.393 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.545 ******* 2020-11-20T13:53:04.399 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.433 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.585 ******* 2020-11-20T13:53:04.439 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-20T13:53:04.440 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.473 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.039) 0:01:54.625 ******* 2020-11-20T13:53:04.479 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.513 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.039) 0:01:54.665 ******* 2020-11-20T13:53:04.518 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-20T13:53:04.520 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.552 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.039) 0:01:54.704 ******* 2020-11-20T13:53:04.559 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.591 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.039) 0:01:54.743 ******* 2020-11-20T13:53:04.597 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-20T13:53:04.599 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.631 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.783 ******* 2020-11-20T13:53:04.638 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.676 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.044) 0:01:54.828 ******* 2020-11-20T13:53:04.681 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-20T13:53:04.683 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.716 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.040) 0:01:54.868 ******* 2020-11-20T13:53:04.722 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:04.757 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:04 +0000 (0:00:00.041) 0:01:54.909 ******* 2020-11-20T13:53:05.179 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-20T13:53:05.180 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-20T13:53:05.213 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.456) 0:01:55.365 ******* 2020-11-20T13:53:05.220 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-11-20T13:53:05.256 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.042) 0:01:55.407 ******* 2020-11-20T13:53:05.513 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-20T13:53:05.514 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:02 UTC", "ActiveEnterTimestampMonotonic": "14697609", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service basic.target ntpdate.service -.mount system.slice sntp.service tmp.mount ntpd.service systemd-journald.socket sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:01 UTC", "AssertTimestampMonotonic": "13581912", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:01 UTC", "ConditionTimestampMonotonic": "13581888", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-20T13:53:05.516 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2405", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:01 UTC", "ExecMainStartTimestampMonotonic": "13783496", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:01 UTC", "InactiveExitTimestampMonotonic": "13597529", "InvocationID": "6b02a2d7fb8d4b0bb14f3416b02aae3f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-20T13:53:05.517 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2405", "MemoryAccounting": "yes", "MemoryCurrent": "2809856", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-20T13:53:05.519 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:02 UTC", "StateChangeTimestampMonotonic": "14697609", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:01 UTC", "WatchdogTimestampMonotonic": "13783506", "WatchdogUSec": "0"}} 2020-11-20T13:53:05.550 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.294) 0:01:55.702 ******* 2020-11-20T13:53:05.558 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-11-20 13:50:21 UTC", "ActiveEnterTimestampMonotonic": "16039154", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service sysinit.target basic.target -.mount systemd-journald.socket sntp.service ntpdate.service ntpd.service tmp.mount system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-11-20 13:50:19 UTC", "AssertTimestampMonotonic": "14706944", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-11-20 13:50:19 UTC", "ConditionTimestampMonotonic": "14706919", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service ntpd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-11-20T13:53:05.560 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2465", "ExecMainStartTimestamp": "Fri 2020-11-20 13:50:21 UTC", "ExecMainStartTimestampMonotonic": "15948893", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-11-20 13:50:19 UTC", "InactiveExitTimestampMonotonic": "14735530", "InvocationID": "4357059a69ee487daf43af056b371035", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-11-20T13:53:05.562 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127426", "LimitNPROCSoft": "127426", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127426", "LimitSIGPENDINGSoft": "127426", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2465", "MemoryAccounting": "yes", "MemoryCurrent": "2703360", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-11-20T13:53:05.564 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-11-20 13:50:21 UTC", "StateChangeTimestampMonotonic": "16039154", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-11-20 13:50:21 UTC", "WatchdogTimestampMonotonic": "15948907", "WatchdogUSec": "0"}} 2020-11-20T13:53:05.593 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.042) 0:01:55.745 ******* 2020-11-20T13:53:05.600 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:05.602 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T13:53:05.634 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.040) 0:01:55.786 ******* 2020-11-20T13:53:05.641 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T13:53:05.675 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.041) 0:01:55.827 ******* 2020-11-20T13:53:05.680 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:05.682 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-20T13:53:05.715 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.039) 0:01:55.867 ******* 2020-11-20T13:53:05.722 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-11-20T13:53:05.755 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.040) 0:01:55.907 ******* 2020-11-20T13:53:05.760 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-20T13:53:05.762 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:05.795 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.039) 0:01:55.947 ******* 2020-11-20T13:53:05.801 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:05.834 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.039) 0:01:55.986 ******* 2020-11-20T13:53:05.840 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-20T13:53:05.842 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:05.875 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.040) 0:01:56.027 ******* 2020-11-20T13:53:05.882 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:05.917 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.041) 0:01:56.068 ******* 2020-11-20T13:53:05.923 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-20T13:53:05.924 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:05.956 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.039) 0:01:56.108 ******* 2020-11-20T13:53:05.962 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:05.998 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:05 +0000 (0:00:00.041) 0:01:56.150 ******* 2020-11-20T13:53:06.149 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T13:53:06.183 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:06 +0000 (0:00:00.185) 0:01:56.335 ******* 2020-11-20T13:53:06.190 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T13:53:06.224 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:06 +0000 (0:00:00.040) 0:01:56.376 ******* 2020-11-20T13:53:06.644 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-20T13:53:06.646 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-20T13:53:06.679 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:06 +0000 (0:00:00.454) 0:01:56.831 ******* 2020-11-20T13:53:06.696 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-11-20T13:53:06.731 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:06 +0000 (0:00:00.052) 0:01:56.883 ******* 2020-11-20T13:53:07.966 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-20T13:53:07.967 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T13:53:08.000 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:07 +0000 (0:00:01.268) 0:01:58.152 ******* 2020-11-20T13:53:08.006 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T13:53:08.041 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.041) 0:01:58.193 ******* 2020-11-20T13:53:08.047 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:08.082 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.040) 0:01:58.234 ******* 2020-11-20T13:53:08.088 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:08.123 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.040) 0:01:58.275 ******* 2020-11-20T13:53:08.129 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-20T13:53:08.131 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-20T13:53:08.164 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.041) 0:01:58.316 ******* 2020-11-20T13:53:08.170 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-20T13:53:08.205 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.041) 0:01:58.357 ******* 2020-11-20T13:53:08.350 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-20T13:53:08.352 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-20T13:53:08.385 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.179) 0:01:58.537 ******* 2020-11-20T13:53:08.402 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-11-20T13:53:08.438 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.052) 0:01:58.590 ******* 2020-11-20T13:53:08.444 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-20T13:53:08.446 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:08.479 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.041) 0:01:58.631 ******* 2020-11-20T13:53:08.486 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:08.521 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.041) 0:01:58.673 ******* 2020-11-20T13:53:08.529 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-20T13:53:08.531 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T13:53:08.564 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.042) 0:01:58.716 ******* 2020-11-20T13:53:08.571 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T13:53:08.606 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.042) 0:01:58.758 ******* 2020-11-20T13:53:08.613 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-20T13:53:08.615 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:08.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.042) 0:01:58.801 ******* 2020-11-20T13:53:08.655 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:08.693 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.043) 0:01:58.844 ******* 2020-11-20T13:53:08.845 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-20T13:53:08.846 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T13:53:08.884 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.191) 0:01:59.036 ******* 2020-11-20T13:53:08.892 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T13:53:08.930 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:08 +0000 (0:00:00.045) 0:01:59.082 ******* 2020-11-20T13:53:09.072 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-20T13:53:09.074 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T13:53:09.107 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.177) 0:01:59.259 ******* 2020-11-20T13:53:09.124 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T13:53:09.158 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.051) 0:01:59.310 ******* 2020-11-20T13:53:09.164 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-20T13:53:09.166 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.198 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.039) 0:01:59.350 ******* 2020-11-20T13:53:09.205 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.240 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.041) 0:01:59.392 ******* 2020-11-20T13:53:09.246 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-20T13:53:09.248 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.259 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.260 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.294 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.054) 0:01:59.446 ******* 2020-11-20T13:53:09.299 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.300 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.300 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.301 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.301 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.350 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.055) 0:01:59.502 ******* 2020-11-20T13:53:09.355 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-20T13:53:09.357 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.358 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi117.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.393 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.043) 0:01:59.545 ******* 2020-11-20T13:53:09.400 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.400 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.402 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi085.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.413 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.414 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T13:53:09.415 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.448 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.054) 0:01:59.600 ******* 2020-11-20T13:53:09.481 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.033) 0:01:59.633 ******* 2020-11-20T13:53:09.487 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.524 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.042) 0:01:59.676 ******* 2020-11-20T13:53:09.530 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-20T13:53:09.531 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.564 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.040) 0:01:59.716 ******* 2020-11-20T13:53:09.570 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.603 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.039) 0:01:59.755 ******* 2020-11-20T13:53:09.608 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-20T13:53:09.610 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.645 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.041) 0:01:59.797 ******* 2020-11-20T13:53:09.650 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.687 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.042) 0:01:59.839 ******* 2020-11-20T13:53:09.693 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-20T13:53:09.695 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.727 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.039) 0:01:59.879 ******* 2020-11-20T13:53:09.733 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.767 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.039) 0:01:59.919 ******* 2020-11-20T13:53:09.773 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-20T13:53:09.774 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.811 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.042) 0:01:59.961 ******* 2020-11-20T13:53:09.815 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.850 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.041) 0:02:00.003 ******* 2020-11-20T13:53:09.856 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T13:53:09.858 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.892 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.041) 0:02:00.044 ******* 2020-11-20T13:53:09.898 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.939 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.046) 0:02:00.091 ******* 2020-11-20T13:53:09.945 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-20T13:53:09.947 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:09.982 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:09 +0000 (0:00:00.043) 0:02:00.134 ******* 2020-11-20T13:53:09.988 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.025 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.042) 0:02:00.177 ******* 2020-11-20T13:53:10.031 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T13:53:10.033 INFO:teuthology.task.ansible.out:skipping: [smithi117.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.066 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.041) 0:02:00.218 ******* 2020-11-20T13:53:10.072 INFO:teuthology.task.ansible.out:skipping: [smithi085.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T13:53:10.107 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.040) 0:02:00.259 ******* 2020-11-20T13:53:10.277 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T13:53:10.279 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004716", "end": "2020-11-20 13:53:10.247936", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 13:53:10.243220", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.330 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004659", "end": "2020-11-20 13:53:10.295964", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 13:53:10.291305", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.454 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003047", "end": "2020-11-20 13:53:10.424303", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 13:53:10.421256", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.496 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003004", "end": "2020-11-20 13:53:10.472727", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 13:53:10.469723", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.632 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002757", "end": "2020-11-20 13:53:10.600368", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:53:10.597611", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.674 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002787", "end": "2020-11-20 13:53:10.646756", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 13:53:10.643969", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.830 INFO:teuthology.task.ansible.out:ok: [smithi117.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.030374", "end": "2020-11-20 13:53:10.801283", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 13:53:10.770909", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.866 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.758) 0:02:01.018 ******* 2020-11-20T13:53:10.871 INFO:teuthology.task.ansible.out:ok: [smithi085.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.020848", "end": "2020-11-20 13:53:10.838850", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 13:53:10.818002", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T13:53:10.907 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.041) 0:02:01.059 ******* 2020-11-20T13:53:10.913 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-20T13:53:10.947 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.039) 0:02:01.099 ******* 2020-11-20T13:53:10.987 INFO:teuthology.task.ansible.out:Friday 20 November 2020 13:53:10 +0000 (0:00:00.039) 0:02:01.139 ******* 2020-11-20T13:53:11.128 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-20T13:53:11.130 INFO:teuthology.task.ansible.out:changed: [smithi117.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:53:11.194 INFO:teuthology.task.ansible.out:changed: [smithi085.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-11-20T13:53:11.287 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-11-20T13:53:11.291 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-20T13:53:11.294 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2020-11-20T13:53:11.296 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi085.front.sepia.ceph.com : ok=115 changed=33 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 smithi117.front.sepia.ceph.com : ok=115 changed=33 unreachable=0 failed=0 skipped=181 rescued=0 ignored=3 Friday 20 November 2020 13:53:11 +0000 (0:00:00.307) 0:02:01.447 ******* =============================================================================== common : nrpe - Remove SELinux policy package -------------------------- 16.59s common : nrpe - Load SELinux policy package ---------------------------- 16.46s testnode : Zap all non-root disks --------------------------------------- 7.69s common : Make sure smartmontools is installed --------------------------- 3.30s testnode : Install packages --------------------------------------------- 2.89s testnode : Ensure ceph packages are not present. ------------------------ 2.15s testnode : Create logical volume(s) ------------------------------------- 1.91s testnode : Ensure ceph dependency packages are not present. ------------- 1.91s testnode : Rebuild rpmdb ------------------------------------------------ 1.77s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.64s testnode : Create xfs filesystem on /dev/vg_nvme/lv_5 ------------------- 1.52s Gathering Facts --------------------------------------------------------- 1.44s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.42s common : nrpe - Set SELinux boolean nagios_run_sudo true ---------------- 1.31s testnode : Install packages via pip ------------------------------------- 1.27s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.26s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.19s testnode : Configure local mirrorlists ---------------------------------- 1.18s common : Instal 2020-11-20T13:53:11.297 INFO:teuthology.task.ansible.out:l Kerberos Packages (RedHat) ----------------------------- 1.16s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 0.92s 2020-11-20T13:53:11.569 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi085.front.sepia.ceph.com')] 2020-11-20T13:53:11.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:53:11.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:53:11.638 INFO:teuthology.orchestra.run.smithi117:> true 2020-11-20T13:53:11.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T13:53:11.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T13:53:11.793 INFO:teuthology.orchestra.run.smithi085:> true 2020-11-20T13:53:11.878 INFO:teuthology.run_tasks:Running task clock... 2020-11-20T13:53:11.896 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-20T13:53:11.897 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T13:53:11.897 INFO:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T13:53:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-20T13:53:11.950 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-20T13:53:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2020-11-20T13:53:12.004 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2020-11-20T13:53:12.021 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-20T13:53:12.039 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-20T13:53:12.101 INFO:teuthology.orchestra.run.smithi117.stderr:bash: ntpq: command not found 2020-11-20T13:53:12.104 INFO:teuthology.orchestra.run.smithi117.stdout:210 Number of sources = 4 2020-11-20T13:53:12.104 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-20T13:53:12.104 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2020-11-20T13:53:12.105 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.105 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.105 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.105 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.106 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T13:53:12.106 INFO:teuthology.orchestra.run.smithi085:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T13:53:12.144 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-11-20T13:53:12.163 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-11-20T13:53:12.203 INFO:teuthology.orchestra.run.smithi085.stderr:sudo: ntpd: command not found 2020-11-20T13:53:12.217 INFO:teuthology.orchestra.run.smithi085.stdout:506 Cannot talk to daemon 2020-11-20T13:53:12.234 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-11-20T13:53:12.251 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-11-20T13:53:12.316 INFO:teuthology.orchestra.run.smithi085.stderr:bash: ntpq: command not found 2020-11-20T13:53:12.320 INFO:teuthology.orchestra.run.smithi085.stdout:210 Number of sources = 4 2020-11-20T13:53:12.320 INFO:teuthology.orchestra.run.smithi085.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-11-20T13:53:12.320 INFO:teuthology.orchestra.run.smithi085.stdout:=============================================================================== 2020-11-20T13:53:12.321 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.321 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.321 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.321 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-11-20T13:53:12.322 INFO:teuthology.run_tasks:Running task install... 2020-11-20T13:53:12.332 DEBUG:teuthology.task.install:project ceph 2020-11-20T13:53:12.333 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}} 2020-11-20T13:53:12.333 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'} 2020-11-20T13:53:12.334 INFO:teuthology.task.install:Using flavor: basic 2020-11-20T13:53:12.341 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-20T13:53:12.342 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-11-20T13:53:12.342 INFO:teuthology.orchestra.run.smithi117:> sudo lsb_release -is 2020-11-20T13:53:12.378 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS 2020-11-20T13:53:12.379 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:12.380 INFO:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2020-11-20T13:53:12.381 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'tag': None, 'wait_for_package': False} 2020-11-20T13:53:12.381 INFO:teuthology.orchestra.run.smithi117:> sudo lsb_release -is 2020-11-20T13:53:12.418 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS 2020-11-20T13:53:12.419 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:12.419 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'tag': None, 'wait_for_package': False} 2020-11-20T13:53:12.420 INFO:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2020-11-20T13:53:12.455 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS 2020-11-20T13:53:12.456 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:12.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:53:12.494 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS 2020-11-20T13:53:12.495 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:53:12.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:53:12.624 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/ 2020-11-20T13:53:12.624 INFO:teuthology.task.install.rpm:Package version is 15.2.5-588.ga7b62340 2020-11-20T13:53:12.657 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/ 2020-11-20T13:53:12.658 INFO:teuthology.task.install.rpm:Package version is 15.2.5-588.ga7b62340 2020-11-20T13:53:12.766 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-20T13:53:12.766 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:53:12.767 INFO:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:12.780 INFO:teuthology.orchestra.run.smithi117.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:12.793 INFO:teuthology.orchestra.run.smithi117.stderr:1+1 records in 2020-11-20T13:53:12.794 INFO:teuthology.orchestra.run.smithi117.stderr:1+1 records out 2020-11-20T13:53:12.794 INFO:teuthology.orchestra.run.smithi117.stderr:589 bytes copied, 3.5537e-05 s, 16.6 MB/s 2020-11-20T13:53:12.795 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-20T13:53:12.795 INFO:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a7b62340ad27b26642bb54ec916a97c7e1f07a34/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-20T13:53:12.802 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/octopus/a7b62340ad27b26642bb54ec916a97c7e1f07a34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-11-20T13:53:12.803 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:53:12.803 INFO:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:12.816 INFO:teuthology.orchestra.run.smithi085.stderr:+ sudo dd of=/etc/yum.repos.d/ceph.repo 2020-11-20T13:53:12.829 INFO:teuthology.orchestra.run.smithi085.stderr:1+1 records in 2020-11-20T13:53:12.830 INFO:teuthology.orchestra.run.smithi085.stderr:1+1 records out 2020-11-20T13:53:12.830 INFO:teuthology.orchestra.run.smithi085.stderr:589 bytes copied, 3.4247e-05 s, 17.2 MB/s 2020-11-20T13:53:12.831 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2020-11-20T13:53:12.831 INFO:teuthology.orchestra.run.smithi085:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a7b62340ad27b26642bb54ec916a97c7e1f07a34/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-11-20T13:53:12.862 INFO:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-20T13:53:12.899 INFO:teuthology.orchestra.run.smithi085:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-11-20T13:53:12.943 INFO:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-20T13:53:12.979 INFO:teuthology.orchestra.run.smithi085:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-11-20T13:53:13.009 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2020-11-20T13:53:13.010 INFO:teuthology.orchestra.run.smithi117:> sudo yum clean all 2020-11-20T13:53:13.046 INFO:teuthology.orchestra.run.smithi085.stdout:check_obsoletes = 1 2020-11-20T13:53:13.047 INFO:teuthology.orchestra.run.smithi085:> sudo yum clean all 2020-11-20T13:53:13.287 INFO:teuthology.orchestra.run.smithi117.stdout:42 files removed 2020-11-20T13:53:13.315 INFO:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-20T13:53:13.323 INFO:teuthology.orchestra.run.smithi085.stdout:42 files removed 2020-11-20T13:53:13.352 INFO:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2020-11-20T13:53:13.916 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 269 kB/s | 69 kB 00:00 2020-11-20T13:53:13.941 INFO:teuthology.orchestra.run.smithi085.stdout:ceph packages for x86_64 280 kB/s | 69 kB 00:00 2020-11-20T13:53:14.154 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 70 kB/s | 15 kB 00:00 2020-11-20T13:53:14.174 INFO:teuthology.orchestra.run.smithi085.stdout:ceph noarch packages 71 kB/s | 15 kB 00:00 2020-11-20T13:53:14.383 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2020-11-20T13:53:14.401 INFO:teuthology.orchestra.run.smithi085.stdout:ceph source packages 7.0 kB/s | 1.4 kB 00:00 2020-11-20T13:53:14.595 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-8 - AppStream 36 MB/s | 7.0 MB 00:00 2020-11-20T13:53:14.625 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS-8 - AppStream 34 MB/s | 7.0 MB 00:00 2020-11-20T13:53:15.790 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-8 - Base 22 MB/s | 2.2 MB 00:00 2020-11-20T13:53:15.802 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS-8 - Base 24 MB/s | 2.2 MB 00:00 2020-11-20T13:53:16.217 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-8 - Extras 88 kB/s | 5.5 kB 00:00 2020-11-20T13:53:16.219 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS-8 - Extras 88 kB/s | 5.5 kB 00:00 2020-11-20T13:53:16.617 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 24 MB/s | 8.3 MB 00:00 2020-11-20T13:53:16.620 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 24 MB/s | 8.3 MB 00:00 2020-11-20T13:53:18.313 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 4.4 MB/s | 23 kB 00:00 2020-11-20T13:53:18.335 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 4.2 MB/s | 23 kB 00:00 2020-11-20T13:53:19.215 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:19.215 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:19.219 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-11-20T13:53:19.219 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-11-20T13:53:19.220 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-11-20T13:53:19.221 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-11-20T13:53:19.253 INFO:teuthology.orchestra.run.smithi085.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:19.254 INFO:teuthology.orchestra.run.smithi085.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2020-11-20T13:53:19.257 INFO:teuthology.orchestra.run.smithi085.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2020-11-20T13:53:19.258 INFO:teuthology.orchestra.run.smithi085.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2020-11-20T13:53:19.259 INFO:teuthology.orchestra.run.smithi085.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2020-11-20T13:53:19.259 INFO:teuthology.orchestra.run.smithi085.stdout:Package xfsprogs-5.0.0-1.el8.x86_64 is already installed. 2020-11-20T13:53:19.350 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2020-11-20T13:53:19.376 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2020-11-20T13:53:19.376 INFO:teuthology.orchestra.run.smithi117.stdout: Package ArchVersion Repository Size 2020-11-20T13:53:19.376 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2020-11-20T13:53:19.377 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2020-11-20T13:53:19.377 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_642:15.2.5-588.ga7b62340.el8 ceph 6.4 k 2020-11-20T13:53:19.377 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_642:15.2.5-588.ga7b62340.el8 ceph 5.6 M 2020-11-20T13:53:19.377 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 577 k 2020-11-20T13:53:19.377 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_642:15.2.5-588.ga7b62340.el8 ceph 205 k 2020-11-20T13:53:19.378 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_642:15.2.5-588.ga7b62340.el8 ceph 1.4 M 2020-11-20T13:53:19.378 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_642:15.2.5-588.ga7b62340.el8 ceph 10 M 2020-11-20T13:53:19.378 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_642:15.2.5-588.ga7b62340.el8 ceph 42 M 2020-11-20T13:53:19.378 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm x86_642:15.2.5-588.ga7b62340.el8 ceph 57 k 2020-11-20T13:53:19.378 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 22 k 2020-11-20T13:53:19.379 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_642:15.2.5-588.ga7b62340.el8 ceph 511 k 2020-11-20T13:53:19.379 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:19.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_642:15.2.5-588.ga7b62340.el8 ceph 163 k 2020-11-20T13:53:19.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_642:15.2.5-588.ga7b62340.el8 ceph 351 k 2020-11-20T13:53:19.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_642:15.2.5-588.ga7b62340.el8 ceph 334 k 2020-11-20T13:53:19.380 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_642:15.2.5-588.ga7b62340.el8 ceph 113 k 2020-11-20T13:53:19.380 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 72 k 2020-11-20T13:53:19.380 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_642:15.2.5-588.ga7b62340.el8 ceph 2.7 M 2020-11-20T13:53:19.380 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:19.380 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 65 k 2020-11-20T13:53:19.381 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 3.5 M 2020-11-20T13:53:19.381 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.6 M 2020-11-20T13:53:19.381 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 27 k 2020-11-20T13:53:19.381 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-20T13:53:19.382 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-20T13:53:19.382 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-20T13:53:19.382 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-20T13:53:19.382 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-20T13:53:19.383 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-20T13:53:19.383 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-20T13:53:19.383 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-20T13:53:19.383 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2020-11-20T13:53:19.383 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.6 M 2020-11-20T13:53:19.384 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_642:15.2.5-588.ga7b62340.el8 ceph 1.9 M 2020-11-20T13:53:19.384 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2020-11-20T13:53:19.384 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 21 M 2020-11-20T13:53:19.384 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_642:15.2.5-588.ga7b62340.el8 ceph 2.0 M 2020-11-20T13:53:19.384 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_642:15.2.5-588.ga7b62340.el8 ceph 4.4 M 2020-11-20T13:53:19.384 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_642:15.2.5-588.ga7b62340.el8 ceph 17 M 2020-11-20T13:53:19.385 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_642:15.2.5-588.ga7b62340.el8 ceph 25 k 2020-11-20T13:53:19.385 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_642:15.2.5-588.ga7b62340.el8 ceph 381 k 2020-11-20T13:53:19.385 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.2 M 2020-11-20T13:53:19.385 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_642:15.2.5-588.ga7b62340.el8 ceph 40 k 2020-11-20T13:53:19.386 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 73 k 2020-11-20T13:53:19.386 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 24 k 2020-11-20T13:53:19.386 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 176 k 2020-11-20T13:53:19.386 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.7 k 2020-11-20T13:53:19.386 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-20T13:53:19.387 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-20T13:53:19.387 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-20T13:53:19.387 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-20T13:53:19.387 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-20T13:53:19.387 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-20T13:53:19.388 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-20T13:53:19.388 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-20T13:53:19.388 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-20T13:53:19.388 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-20T13:53:19.388 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-20T13:53:19.389 INFO:teuthology.orchestra.run.smithi117.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-20T13:53:19.389 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-20T13:53:19.389 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-20T13:53:19.389 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-20T13:53:19.389 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-20T13:53:19.390 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-20T13:53:19.390 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-20T13:53:19.390 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-20T13:53:19.390 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-20T13:53:19.390 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-20T13:53:19.391 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-20T13:53:19.391 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-20T13:53:19.391 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-20T13:53:19.391 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-20T13:53:19.391 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-20T13:53:19.392 INFO:teuthology.orchestra.run.smithi117.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-20T13:53:19.392 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-20T13:53:19.392 INFO:teuthology.orchestra.run.smithi117.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-20T13:53:19.392 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-20T13:53:19.393 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-20T13:53:19.393 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-20T13:53:19.393 INFO:teuthology.orchestra.run.smithi117.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-20T13:53:19.393 INFO:teuthology.orchestra.run.smithi117.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-20T13:53:19.393 INFO:teuthology.orchestra.run.smithi117.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-20T13:53:19.394 INFO:teuthology.orchestra.run.smithi117.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-20T13:53:19.394 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-20T13:53:19.394 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-20T13:53:19.394 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-20T13:53:19.394 INFO:teuthology.orchestra.run.smithi117.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-20T13:53:19.395 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-20T13:53:19.395 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-20T13:53:19.395 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-20T13:53:19.395 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-20T13:53:19.395 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-20T13:53:19.396 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-20T13:53:19.396 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-20T13:53:19.396 INFO:teuthology.orchestra.run.smithi117.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-20T13:53:19.396 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-20T13:53:19.396 INFO:teuthology.orchestra.run.smithi117.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-20T13:53:19.397 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-20T13:53:19.397 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-20T13:53:19.397 INFO:teuthology.orchestra.run.smithi117.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-20T13:53:19.397 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-20T13:53:19.398 INFO:teuthology.orchestra.run.smithi117.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-20T13:53:19.398 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-20T13:53:19.398 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-20T13:53:19.398 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-20T13:53:19.399 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-20T13:53:19.399 INFO:teuthology.orchestra.run.smithi117.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-20T13:53:19.399 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-20T13:53:19.399 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-20T13:53:19.399 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-20T13:53:19.399 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-20T13:53:19.400 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-20T13:53:19.400 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-20T13:53:19.400 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-20T13:53:19.400 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-20T13:53:19.400 INFO:teuthology.orchestra.run.smithi117.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-20T13:53:19.401 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-20T13:53:19.401 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-20T13:53:19.401 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-20T13:53:19.401 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-20T13:53:19.401 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-20T13:53:19.402 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-20T13:53:19.402 INFO:teuthology.orchestra.run.smithi117.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-20T13:53:19.402 INFO:teuthology.orchestra.run.smithi117.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-20T13:53:19.402 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-20T13:53:19.403 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-20T13:53:19.403 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-20T13:53:19.403 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-20T13:53:19.403 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-20T13:53:19.403 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2020-11-20T13:53:19.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-20T13:53:19.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-20T13:53:19.404 INFO:teuthology.orchestra.run.smithi117.stdout:Downgrading: 2020-11-20T13:53:19.404 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-20T13:53:19.404 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-20T13:53:19.405 INFO:teuthology.orchestra.run.smithi117.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-20T13:53:19.405 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-20T13:53:19.405 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-20T13:53:19.405 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:53:19.405 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2020-11-20T13:53:19.406 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2020-11-20T13:53:19.406 INFO:teuthology.orchestra.run.smithi117.stdout:Install 126 Packages 2020-11-20T13:53:19.406 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 2 Packages 2020-11-20T13:53:19.406 INFO:teuthology.orchestra.run.smithi117.stdout:Downgrade 5 Packages 2020-11-20T13:53:19.406 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:53:19.407 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2020-11-20T13:53:19.407 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 185 M 2020-11-20T13:53:19.408 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2020-11-20T13:53:19.414 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2020-11-20T13:53:19.414 INFO:teuthology.orchestra.run.smithi085.stdout: Package ArchVersion Repository Size 2020-11-20T13:53:19.414 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2020-11-20T13:53:19.414 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2020-11-20T13:53:19.415 INFO:teuthology.orchestra.run.smithi085.stdout: ceph x86_642:15.2.5-588.ga7b62340.el8 ceph 6.4 k 2020-11-20T13:53:19.415 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base x86_642:15.2.5-588.ga7b62340.el8 ceph 5.6 M 2020-11-20T13:53:19.415 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 577 k 2020-11-20T13:53:19.415 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache x86_642:15.2.5-588.ga7b62340.el8 ceph 205 k 2020-11-20T13:53:19.415 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr x86_642:15.2.5-588.ga7b62340.el8 ceph 1.4 M 2020-11-20T13:53:19.416 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw x86_642:15.2.5-588.ga7b62340.el8 ceph 10 M 2020-11-20T13:53:19.416 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test x86_642:15.2.5-588.ga7b62340.el8 ceph 42 M 2020-11-20T13:53:19.416 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm x86_642:15.2.5-588.ga7b62340.el8 ceph 57 k 2020-11-20T13:53:19.416 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 22 k 2020-11-20T13:53:19.417 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2 x86_642:15.2.5-588.ga7b62340.el8 ceph 511 k 2020-11-20T13:53:19.417 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:19.417 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs x86_642:15.2.5-588.ga7b62340.el8 ceph 163 k 2020-11-20T13:53:19.417 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados x86_642:15.2.5-588.ga7b62340.el8 ceph 351 k 2020-11-20T13:53:19.417 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd x86_642:15.2.5-588.ga7b62340.el8 ceph 334 k 2020-11-20T13:53:19.418 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw x86_642:15.2.5-588.ga7b62340.el8 ceph 113 k 2020-11-20T13:53:19.418 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse x86_642:15.2.5-588.ga7b62340.el8 ceph 72 k 2020-11-20T13:53:19.418 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror x86_642:15.2.5-588.ga7b62340.el8 ceph 2.7 M 2020-11-20T13:53:19.418 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd x86_642:15.2.5-588.ga7b62340.el8 ceph 141 k 2020-11-20T13:53:19.419 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 65 k 2020-11-20T13:53:19.419 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 3.5 M 2020-11-20T13:53:19.419 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.6 M 2020-11-20T13:53:19.419 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 27 k 2020-11-20T13:53:19.419 INFO:teuthology.orchestra.run.smithi085.stdout: bison x86_643.0.4-10.el8 CentOS-AppStream 688 k 2020-11-20T13:53:19.420 INFO:teuthology.orchestra.run.smithi085.stdout: flex x86_642.6.1-9.el8 CentOS-AppStream 320 k 2020-11-20T13:53:19.420 INFO:teuthology.orchestra.run.smithi085.stdout: libtool x86_642.4.6-25.el8 CentOS-AppStream 709 k 2020-11-20T13:53:19.420 INFO:teuthology.orchestra.run.smithi085.stdout: elfutils-libelf-devel x86_640.176-5.el8 CentOS-Base 54 k 2020-11-20T13:53:19.420 INFO:teuthology.orchestra.run.smithi085.stdout: libaio-devel x86_640.3.112-1.el8 CentOS-Base 19 k 2020-11-20T13:53:19.420 INFO:teuthology.orchestra.run.smithi085.stdout: libuuid-devel x86_642.32.1-17.el8 CentOS-Base 95 k 2020-11-20T13:53:19.421 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-devel x86_641:1.1.1c-2.el8_1.1 CentOS-Base 2.3 M 2020-11-20T13:53:19.421 INFO:teuthology.orchestra.run.smithi085.stdout: xfsprogs-devel x86_645.0.0-1.el8 CentOS-Base 69 k 2020-11-20T13:53:19.421 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrading: 2020-11-20T13:53:19.421 INFO:teuthology.orchestra.run.smithi085.stdout: librados2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.6 M 2020-11-20T13:53:19.422 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1 x86_642:15.2.5-588.ga7b62340.el8 ceph 1.9 M 2020-11-20T13:53:19.422 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2020-11-20T13:53:19.422 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 21 M 2020-11-20T13:53:19.422 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds x86_642:15.2.5-588.ga7b62340.el8 ceph 2.0 M 2020-11-20T13:53:19.423 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon x86_642:15.2.5-588.ga7b62340.el8 ceph 4.4 M 2020-11-20T13:53:19.423 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd x86_642:15.2.5-588.ga7b62340.el8 ceph 17 M 2020-11-20T13:53:19.423 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux x86_642:15.2.5-588.ga7b62340.el8 ceph 25 k 2020-11-20T13:53:19.423 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1 x86_642:15.2.5-588.ga7b62340.el8 ceph 381 k 2020-11-20T13:53:19.423 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2 x86_642:15.2.5-588.ga7b62340.el8 ceph 3.2 M 2020-11-20T13:53:19.423 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse x86_642:15.2.5-588.ga7b62340.el8 ceph 40 k 2020-11-20T13:53:19.424 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common x86_642:15.2.5-588.ga7b62340.el8 ceph 73 k 2020-11-20T13:53:19.424 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 24 k 2020-11-20T13:53:19.424 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 176 k 2020-11-20T13:53:19.424 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts noarch2:15.2.5-588.ga7b62340.el8 ceph-noarch 8.7 k 2020-11-20T13:53:19.425 INFO:teuthology.orchestra.run.smithi085.stdout: automake noarch1.16.1-6.el8 CentOS-AppStream 713 k 2020-11-20T13:53:19.425 INFO:teuthology.orchestra.run.smithi085.stdout: jq x86_641.5-12.el8 CentOS-AppStream 161 k 2020-11-20T13:53:19.425 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka x86_640.11.4-1.el8 CentOS-AppStream 353 k 2020-11-20T13:53:19.426 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust x86_642.8.1-11.el8 CentOS-AppStream 259 k 2020-11-20T13:53:19.426 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma x86_646.8.2-1.el8 CentOS-AppStream 188 k 2020-11-20T13:53:19.426 INFO:teuthology.orchestra.run.smithi085.stdout: openblas x86_640.3.3-2.el8 CentOS-AppStream 4.3 M 2020-11-20T13:53:19.426 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads x86_640.3.3-2.el8 CentOS-AppStream 4.4 M 2020-11-20T13:53:19.426 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel noarch2.5.1-5.el8 CentOS-AppStream 4.8 M 2020-11-20T13:53:19.427 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2 noarch2.10.1-2.el8_0 CentOS-AppStream 538 k 2020-11-20T13:53:19.427 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch noarch1.21-2.el8 CentOS-AppStream 27 k 2020-11-20T13:53:19.427 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer noarch1.10-11.el8 CentOS-AppStream 20 k 2020-11-20T13:53:19.427 INFO:teuthology.orchestra.run.smithi085.stdout: python3-lxml x86_644.2.3-1.el8 CentOS-AppStream 1.5 M 2020-11-20T13:53:19.427 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako noarch1.0.6-13.el8 CentOS-AppStream 157 k 2020-11-20T13:53:19.428 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe x86_640.23-19.el8 CentOS-AppStream 39 k 2020-11-20T13:53:19.428 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy x86_641:1.14.3-9.el8 CentOS-AppStream 3.7 M 2020-11-20T13:53:19.428 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 CentOS-AppStream 225 k 2020-11-20T13:53:19.428 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable noarch0.7.2-14.el8 CentOS-AppStream 44 k 2020-11-20T13:53:19.428 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1 noarch0.3.7-6.el8 CentOS-AppStream 126 k 2020-11-20T13:53:19.429 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 CentOS-AppStream 110 k 2020-11-20T13:53:19.429 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz noarch2017.2-9.el8 CentOS-AppStream 54 k 2020-11-20T13:53:19.429 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy x86_641.0.0-20.module_el8.1.0+245+c39af44f CentOS-AppStream 15 M 2020-11-20T13:53:19.429 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webencodings noarch0.5.1-6.el8 CentOS-AppStream 27 k 2020-11-20T13:53:19.430 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug noarch0.12.2-4.el8 CentOS-AppStream 457 k 2020-11-20T13:53:19.430 INFO:teuthology.orchestra.run.smithi085.stdout: socat x86_641.7.3.2-6.el8 CentOS-AppStream 298 k 2020-11-20T13:53:19.430 INFO:teuthology.orchestra.run.smithi085.stdout: fuse x86_642.9.7-12.el8 CentOS-Base 82 k 2020-11-20T13:53:19.430 INFO:teuthology.orchestra.run.smithi085.stdout: fuse-common x86_643.2.1-12.el8 CentOS-Base 21 k 2020-11-20T13:53:19.430 INFO:teuthology.orchestra.run.smithi085.stdout: keyutils-libs-devel x86_641.5.10-6.el8 CentOS-Base 48 k 2020-11-20T13:53:19.430 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-devel x86_641.17-9.el8 CentOS-Base 548 k 2020-11-20T13:53:19.431 INFO:teuthology.orchestra.run.smithi085.stdout: libcom_err-devel x86_641.44.6-3.el8 CentOS-Base 38 k 2020-11-20T13:53:19.431 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran x86_648.3.1-4.5.el8 CentOS-Base 638 k 2020-11-20T13:53:19.431 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath x86_648.3.1-4.5.el8 CentOS-Base 169 k 2020-11-20T13:53:19.431 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq x86_640.9.0-1.el8 CentOS-Base 47 k 2020-11-20T13:53:19.431 INFO:teuthology.orchestra.run.smithi085.stdout: libselinux-devel x86_642.9-2.1.el8 CentOS-Base 199 k 2020-11-20T13:53:19.432 INFO:teuthology.orchestra.run.smithi085.stdout: libsepol-devel x86_642.9-1.el8 CentOS-Base 86 k 2020-11-20T13:53:19.432 INFO:teuthology.orchestra.run.smithi085.stdout: libverto-devel x86_640.3.0-5.el8 CentOS-Base 18 k 2020-11-20T13:53:19.432 INFO:teuthology.orchestra.run.smithi085.stdout: libxslt x86_641.1.32-3.el8 CentOS-Base 249 k 2020-11-20T13:53:19.432 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-devel x86_6410.32-1.el8 CentOS-Base 605 k 2020-11-20T13:53:19.432 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf16 x86_6410.32-1.el8 CentOS-Base 228 k 2020-11-20T13:53:19.433 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf32 x86_6410.32-1.el8 CentOS-Base 220 k 2020-11-20T13:53:19.433 INFO:teuthology.orchestra.run.smithi085.stdout: python3-chardet noarch3.0.4-7.el8 CentOS-Base 195 k 2020-11-20T13:53:19.433 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt noarch1.6.1-2.el8 CentOS-Base 43 k 2020-11-20T13:53:19.433 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib noarch2.1.0-1.el8 CentOS-Base 155 k 2020-11-20T13:53:19.434 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pysocks noarch1.6.8-3.el8 CentOS-Base 34 k 2020-11-20T13:53:19.434 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests noarch2.20.0-2.1.el8_1 CentOS-Base 123 k 2020-11-20T13:53:19.434 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 CentOS-Base 43 k 2020-11-20T13:53:19.434 INFO:teuthology.orchestra.run.smithi085.stdout: python3-urllib3 noarch1.24.2-2.el8 CentOS-Base 176 k 2020-11-20T13:53:19.434 INFO:teuthology.orchestra.run.smithi085.stdout: zlib-devel x86_641.2.11-10.el8 CentOS-Base 56 k 2020-11-20T13:53:19.435 INFO:teuthology.orchestra.run.smithi085.stdout: fmt x86_646.2.1-1.el8 epel 116 k 2020-11-20T13:53:19.435 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs x86_642.7-6.el8 epel 315 k 2020-11-20T13:53:19.435 INFO:teuthology.orchestra.run.smithi085.stdout: leveldb x86_641.22-1.el8 epel 181 k 2020-11-20T13:53:19.435 INFO:teuthology.orchestra.run.smithi085.stdout: liboath x86_642.6.2-3.el8 epel 59 k 2020-11-20T13:53:19.435 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind x86_641.3.1-3.el8 epel 75 k 2020-11-20T13:53:19.436 INFO:teuthology.orchestra.run.smithi085.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-11-20T13:53:19.436 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-11-20T13:53:19.436 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-11-20T13:53:19.436 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-11-20T13:53:19.436 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-11-20T13:53:19.437 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot noarch8.2.1-1.el8 epel 145 k 2020-11-20T13:53:19.437 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy noarch18.4.0-1.el8 epel 384 k 2020-11-20T13:53:19.437 INFO:teuthology.orchestra.run.smithi085.stdout: python3-execnet noarch1.7.1-1.el8 epel 355 k 2020-11-20T13:53:19.437 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-11-20T13:53:19.437 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco noarch6.2-6.el8 epel 11 k 2020-11-20T13:53:19.438 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools noarch2.0-4.el8 epel 18 k 2020-11-20T13:53:19.438 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes noarch11.0.0-2.el8 epel 1.2 M 2020-11-20T13:53:19.438 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils noarch0.3.5-11.el8 epel 49 k 2020-11-20T13:53:19.438 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools noarch7.2.0-3.el8 epel 59 k 2020-11-20T13:53:19.438 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan noarch1.3.2-9.el8 epel 283 k 2020-11-20T13:53:19.439 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend noarch2.6-1.el8 epel 16 k 2020-11-20T13:53:19.439 INFO:teuthology.orchestra.run.smithi085.stdout: python3-remoto noarch1.1.4-4.el8 epel 48 k 2020-11-20T13:53:19.439 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru noarch0.7-6.el8 epel 34 k 2020-11-20T13:53:19.439 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes noarch2.4.1-12.el8 epel 196 k 2020-11-20T13:53:19.439 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa noarch3.4.2-15.el8 epel 73 k 2020-11-20T13:53:19.440 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric noarch0.8.1-17.el8 epel 19 k 2020-11-20T13:53:19.440 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch noarch3.4.0.3-18.el8 epel 24 k 2020-11-20T13:53:19.440 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora noarch1.14.1-5.el8 epel 29 k 2020-11-20T13:53:19.440 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme noarch0.6.0-4.el8 epel 27 k 2020-11-20T13:53:19.440 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-11-20T13:53:19.441 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-11-20T13:53:19.441 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-11-20T13:53:19.441 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-11-20T13:53:19.441 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile noarch2.0-2.el8 epel 23 k 2020-11-20T13:53:19.441 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet x86_641.6.1-11.el8 epel 69 k 2020-11-20T13:53:19.442 INFO:teuthology.orchestra.run.smithi085.stdout:Installing weak dependencies: 2020-11-20T13:53:19.442 INFO:teuthology.orchestra.run.smithi085.stdout: python3-html5lib noarch1:0.999999999-6.el8 CentOS-AppStream 214 k 2020-11-20T13:53:19.442 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-11-20T13:53:19.442 INFO:teuthology.orchestra.run.smithi085.stdout:Downgrading: 2020-11-20T13:53:19.442 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-libs x86_641.17-9.el8 CentOS-Base 851 k 2020-11-20T13:53:19.443 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-workstation x86_641.17-9.el8 CentOS-Base 940 k 2020-11-20T13:53:19.443 INFO:teuthology.orchestra.run.smithi085.stdout: libkadm5 x86_641.17-9.el8 CentOS-Base 184 k 2020-11-20T13:53:19.443 INFO:teuthology.orchestra.run.smithi085.stdout: openssl x86_641:1.1.1c-2.el8_1.1 CentOS-Base 686 k 2020-11-20T13:53:19.443 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-libs x86_641:1.1.1c-2.el8_1.1 CentOS-Base 1.5 M 2020-11-20T13:53:19.443 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:53:19.443 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2020-11-20T13:53:19.444 INFO:teuthology.orchestra.run.smithi085.stdout:================================================================================ 2020-11-20T13:53:19.444 INFO:teuthology.orchestra.run.smithi085.stdout:Install 126 Packages 2020-11-20T13:53:19.444 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrade 2 Packages 2020-11-20T13:53:19.444 INFO:teuthology.orchestra.run.smithi085.stdout:Downgrade 5 Packages 2020-11-20T13:53:19.444 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:53:19.445 INFO:teuthology.orchestra.run.smithi117.stdout:(1/133): krb5-libs-1.17-9.el8.x86_64.rpm 21 MB/s | 851 kB 00:00 2020-11-20T13:53:19.445 INFO:teuthology.orchestra.run.smithi117.stdout:(2/133): libkadm5-1.17-9.el8.x86_64.rpm 4.1 MB/s | 184 kB 00:00 2020-11-20T13:53:19.446 INFO:teuthology.orchestra.run.smithi117.stdout:(3/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 50 MB/s | 686 kB 00:00 2020-11-20T13:53:19.446 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 185 M 2020-11-20T13:53:19.446 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2020-11-20T13:53:19.456 INFO:teuthology.orchestra.run.smithi117.stdout:(4/133): krb5-workstation-1.17-9.el8.x86_64.rpm 14 MB/s | 940 kB 00:00 2020-11-20T13:53:19.462 INFO:teuthology.orchestra.run.smithi117.stdout:(5/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 55 MB/s | 1.5 MB 00:00 2020-11-20T13:53:19.473 INFO:teuthology.orchestra.run.smithi085.stdout:(1/133): krb5-libs-1.17-9.el8.x86_64.rpm 19 MB/s | 851 kB 00:00 2020-11-20T13:53:19.474 INFO:teuthology.orchestra.run.smithi085.stdout:(2/133): libkadm5-1.17-9.el8.x86_64.rpm 4.1 MB/s | 184 kB 00:00 2020-11-20T13:53:19.488 INFO:teuthology.orchestra.run.smithi085.stdout:(3/133): openssl-1.1.1c-2.el8_1.1.x86_64.rpm 44 MB/s | 686 kB 00:00 2020-11-20T13:53:19.496 INFO:teuthology.orchestra.run.smithi085.stdout:(4/133): krb5-workstation-1.17-9.el8.x86_64.rpm 14 MB/s | 940 kB 00:00 2020-11-20T13:53:19.508 INFO:teuthology.orchestra.run.smithi085.stdout:(5/133): openssl-libs-1.1.1c-2.el8_1.1.x86_64.r 42 MB/s | 1.5 MB 00:00 2020-11-20T13:53:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:(6/133): ceph-15.2.5-588.ga7b62340.el8.x86_64.r 57 kB/s | 6.4 kB 00:00 2020-11-20T13:53:19.590 INFO:teuthology.orchestra.run.smithi085.stdout:(6/133): ceph-15.2.5-588.ga7b62340.el8.x86_64.r 64 kB/s | 6.4 kB 00:00 2020-11-20T13:53:19.767 INFO:teuthology.orchestra.run.smithi117.stdout:(7/133): ceph-fuse-15.2.5-588.ga7b62340.el8.x86 2.7 MB/s | 577 kB 00:00 2020-11-20T13:53:19.863 INFO:teuthology.orchestra.run.smithi085.stdout:(7/133): ceph-fuse-15.2.5-588.ga7b62340.el8.x86 2.1 MB/s | 577 kB 00:00 2020-11-20T13:53:19.906 INFO:teuthology.orchestra.run.smithi117.stdout:(8/133): ceph-immutable-object-cache-15.2.5-588 1.4 MB/s | 205 kB 00:00 2020-11-20T13:53:19.981 INFO:teuthology.orchestra.run.smithi085.stdout:(8/133): ceph-immutable-object-cache-15.2.5-588 1.7 MB/s | 205 kB 00:00 2020-11-20T13:53:20.576 INFO:teuthology.orchestra.run.smithi117.stdout:(9/133): ceph-mds-15.2.5-588.ga7b62340.el8.x86_ 3.0 MB/s | 2.0 MB 00:00 2020-11-20T13:53:20.594 INFO:teuthology.orchestra.run.smithi117.stdout:(10/133): ceph-base-15.2.5-588.ga7b62340.el8.x8 4.9 MB/s | 5.6 MB 00:01 2020-11-20T13:53:20.782 INFO:teuthology.orchestra.run.smithi085.stdout:(9/133): ceph-mds-15.2.5-588.ga7b62340.el8.x86_ 2.5 MB/s | 2.0 MB 00:00 2020-11-20T13:53:20.846 INFO:teuthology.orchestra.run.smithi085.stdout:(10/133): ceph-base-15.2.5-588.ga7b62340.el8.x8 4.1 MB/s | 5.6 MB 00:01 2020-11-20T13:53:21.124 INFO:teuthology.orchestra.run.smithi117.stdout:(11/133): ceph-mgr-15.2.5-588.ga7b62340.el8.x86 2.6 MB/s | 1.4 MB 00:00 2020-11-20T13:53:21.338 INFO:teuthology.orchestra.run.smithi085.stdout:(11/133): ceph-mgr-15.2.5-588.ga7b62340.el8.x86 2.6 MB/s | 1.4 MB 00:00 2020-11-20T13:53:21.536 INFO:teuthology.orchestra.run.smithi117.stdout:(12/133): ceph-mon-15.2.5-588.ga7b62340.el8.x86 4.6 MB/s | 4.4 MB 00:00 2020-11-20T13:53:21.795 INFO:teuthology.orchestra.run.smithi085.stdout:(12/133): ceph-mon-15.2.5-588.ga7b62340.el8.x86 4.6 MB/s | 4.4 MB 00:00 2020-11-20T13:53:21.908 INFO:teuthology.orchestra.run.smithi117.stdout:(13/133): ceph-common-15.2.5-588.ga7b62340.el8. 8.6 MB/s | 21 MB 00:02 2020-11-20T13:53:22.038 INFO:teuthology.orchestra.run.smithi117.stdout:(14/133): ceph-selinux-15.2.5-588.ga7b62340.el8 191 kB/s | 25 kB 00:00 2020-11-20T13:53:23.622 INFO:teuthology.orchestra.run.smithi117.stdout:(15/133): ceph-radosgw-15.2.5-588.ga7b62340.el8 4.9 MB/s | 10 MB 00:02 2020-11-20T13:53:23.693 INFO:teuthology.orchestra.run.smithi117.stdout:(16/133): cephadm-15.2.5-588.ga7b62340.el8.x86_ 805 kB/s | 57 kB 00:00 2020-11-20T13:53:23.782 INFO:teuthology.orchestra.run.smithi117.stdout:(17/133): libcephfs-devel-15.2.5-588.ga7b62340. 245 kB/s | 22 kB 00:00 2020-11-20T13:53:23.850 INFO:teuthology.orchestra.run.smithi085.stdout:(13/133): ceph-radosgw-15.2.5-588.ga7b62340.el8 4.9 MB/s | 10 MB 00:02 2020-11-20T13:53:23.932 INFO:teuthology.orchestra.run.smithi117.stdout:(18/133): libcephfs2-15.2.5-588.ga7b62340.el8.x 3.3 MB/s | 511 kB 00:00 2020-11-20T13:53:23.988 INFO:teuthology.orchestra.run.smithi117.stdout:(19/133): librados-devel-15.2.5-588.ga7b62340.e 2.4 MB/s | 141 kB 00:00 2020-11-20T13:53:24.014 INFO:teuthology.orchestra.run.smithi085.stdout:(14/133): ceph-common-15.2.5-588.ga7b62340.el8. 4.6 MB/s | 21 MB 00:04 2020-11-20T13:53:24.014 INFO:teuthology.orchestra.run.smithi085.stdout:(15/133): ceph-selinux-15.2.5-588.ga7b62340.el8 151 kB/s | 25 kB 00:00 2020-11-20T13:53:24.071 INFO:teuthology.orchestra.run.smithi117.stdout:(20/133): libradosstriper1-15.2.5-588.ga7b62340 4.5 MB/s | 381 kB 00:00 2020-11-20T13:53:24.361 INFO:teuthology.orchestra.run.smithi117.stdout:(21/133): librgw2-15.2.5-588.ga7b62340.el8.x86_ 11 MB/s | 3.2 MB 00:00 2020-11-20T13:53:24.400 INFO:teuthology.orchestra.run.smithi117.stdout:(22/133): python3-ceph-argparse-15.2.5-588.ga7b 1.0 MB/s | 40 kB 00:00 2020-11-20T13:53:24.431 INFO:teuthology.orchestra.run.smithi117.stdout:(23/133): python3-ceph-common-15.2.5-588.ga7b62 2.3 MB/s | 73 kB 00:00 2020-11-20T13:53:24.458 INFO:teuthology.orchestra.run.smithi117.stdout:(24/133): python3-cephfs-15.2.5-588.ga7b62340.e 6.0 MB/s | 163 kB 00:00 2020-11-20T13:53:24.516 INFO:teuthology.orchestra.run.smithi117.stdout:(25/133): python3-rados-15.2.5-588.ga7b62340.el 5.9 MB/s | 351 kB 00:00 2020-11-20T13:53:24.547 INFO:teuthology.orchestra.run.smithi117.stdout:(26/133): python3-rbd-15.2.5-588.ga7b62340.el8. 11 MB/s | 334 kB 00:00 2020-11-20T13:53:24.574 INFO:teuthology.orchestra.run.smithi117.stdout:(27/133): python3-rgw-15.2.5-588.ga7b62340.el8. 4.1 MB/s | 113 kB 00:00 2020-11-20T13:53:24.599 INFO:teuthology.orchestra.run.smithi117.stdout:(28/133): rbd-fuse-15.2.5-588.ga7b62340.el8.x86 2.8 MB/s | 72 kB 00:00 2020-11-20T13:53:25.244 INFO:teuthology.orchestra.run.smithi085.stdout:(16/133): cephadm-15.2.5-588.ga7b62340.el8.x86_ 46 kB/s | 57 kB 00:01 2020-11-20T13:53:25.268 INFO:teuthology.orchestra.run.smithi117.stdout:(29/133): rbd-mirror-15.2.5-588.ga7b62340.el8.x 4.1 MB/s | 2.7 MB 00:00 2020-11-20T13:53:25.292 INFO:teuthology.orchestra.run.smithi085.stdout:(17/133): libcephfs-devel-15.2.5-588.ga7b62340. 482 kB/s | 22 kB 00:00 2020-11-20T13:53:25.378 INFO:teuthology.orchestra.run.smithi117.stdout:(30/133): ceph-osd-15.2.5-588.ga7b62340.el8.x86 3.9 MB/s | 17 MB 00:04 2020-11-20T13:53:25.380 INFO:teuthology.orchestra.run.smithi117.stdout:(31/133): rbd-nbd-15.2.5-588.ga7b62340.el8.x86_ 1.3 MB/s | 141 kB 00:00 2020-11-20T13:53:25.421 INFO:teuthology.orchestra.run.smithi085.stdout:(18/133): libcephfs2-15.2.5-588.ga7b62340.el8.x 3.9 MB/s | 511 kB 00:00 2020-11-20T13:53:25.442 INFO:teuthology.orchestra.run.smithi117.stdout:(32/133): ceph-grafana-dashboards-15.2.5-588.ga 379 kB/s | 24 kB 00:00 2020-11-20T13:53:25.454 INFO:teuthology.orchestra.run.smithi117.stdout:(33/133): ceph-mgr-cephadm-15.2.5-588.ga7b62340 881 kB/s | 65 kB 00:00 2020-11-20T13:53:25.477 INFO:teuthology.orchestra.run.smithi085.stdout:(19/133): librados-devel-15.2.5-588.ga7b62340.e 2.5 MB/s | 141 kB 00:00 2020-11-20T13:53:25.617 INFO:teuthology.orchestra.run.smithi085.stdout:(20/133): libradosstriper1-15.2.5-588.ga7b62340 2.7 MB/s | 381 kB 00:00 2020-11-20T13:53:26.080 INFO:teuthology.orchestra.run.smithi085.stdout:(21/133): librgw2-15.2.5-588.ga7b62340.el8.x86_ 7.0 MB/s | 3.2 MB 00:00 2020-11-20T13:53:26.110 INFO:teuthology.orchestra.run.smithi085.stdout:(22/133): python3-ceph-argparse-15.2.5-588.ga7b 1.3 MB/s | 40 kB 00:00 2020-11-20T13:53:26.136 INFO:teuthology.orchestra.run.smithi085.stdout:(23/133): python3-ceph-common-15.2.5-588.ga7b62 2.8 MB/s | 73 kB 00:00 2020-11-20T13:53:26.169 INFO:teuthology.orchestra.run.smithi117.stdout:(34/133): ceph-test-15.2.5-588.ga7b62340.el8.x8 10 MB/s | 42 MB 00:04 2020-11-20T13:53:26.176 INFO:teuthology.orchestra.run.smithi085.stdout:(24/133): python3-cephfs-15.2.5-588.ga7b62340.e 4.0 MB/s | 163 kB 00:00 2020-11-20T13:53:26.227 INFO:teuthology.orchestra.run.smithi117.stdout:(35/133): ceph-mgr-modules-core-15.2.5-588.ga7b 3.0 MB/s | 176 kB 00:00 2020-11-20T13:53:26.240 INFO:teuthology.orchestra.run.smithi085.stdout:(25/133): python3-rados-15.2.5-588.ga7b62340.el 5.3 MB/s | 351 kB 00:00 2020-11-20T13:53:26.273 INFO:teuthology.orchestra.run.smithi117.stdout:(36/133): ceph-mgr-rook-15.2.5-588.ga7b62340.el 594 kB/s | 27 kB 00:00 2020-11-20T13:53:26.289 INFO:teuthology.orchestra.run.smithi085.stdout:(26/133): python3-rbd-15.2.5-588.ga7b62340.el8. 6.7 MB/s | 334 kB 00:00 2020-11-20T13:53:26.320 INFO:teuthology.orchestra.run.smithi117.stdout:(37/133): ceph-prometheus-alerts-15.2.5-588.ga7 187 kB/s | 8.7 kB 00:00 2020-11-20T13:53:26.328 INFO:teuthology.orchestra.run.smithi085.stdout:(27/133): python3-rgw-15.2.5-588.ga7b62340.el8. 2.9 MB/s | 113 kB 00:00 2020-11-20T13:53:26.356 INFO:teuthology.orchestra.run.smithi117.stdout:(38/133): automake-1.16.1-6.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2020-11-20T13:53:26.365 INFO:teuthology.orchestra.run.smithi085.stdout:(28/133): rbd-fuse-15.2.5-588.ga7b62340.el8.x86 1.9 MB/s | 72 kB 00:00 2020-11-20T13:53:26.365 INFO:teuthology.orchestra.run.smithi117.stdout:(39/133): bison-3.0.4-10.el8.x86_64.rpm 79 MB/s | 688 kB 00:00 2020-11-20T13:53:26.369 INFO:teuthology.orchestra.run.smithi117.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 73 MB/s | 320 kB 00:00 2020-11-20T13:53:26.372 INFO:teuthology.orchestra.run.smithi117.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 61 MB/s | 161 kB 00:00 2020-11-20T13:53:26.377 INFO:teuthology.orchestra.run.smithi117.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 74 MB/s | 353 kB 00:00 2020-11-20T13:53:26.386 INFO:teuthology.orchestra.run.smithi117.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 80 MB/s | 709 kB 00:00 2020-11-20T13:53:26.390 INFO:teuthology.orchestra.run.smithi117.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 67 MB/s | 259 kB 00:00 2020-11-20T13:53:26.393 INFO:teuthology.orchestra.run.smithi117.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 65 MB/s | 188 kB 00:00 2020-11-20T13:53:26.443 INFO:teuthology.orchestra.run.smithi117.stdout:(46/133): openblas-0.3.3-2.el8.x86_64.rpm 86 MB/s | 4.3 MB 00:00 2020-11-20T13:53:26.459 INFO:teuthology.orchestra.run.smithi117.stdout:(47/133): ceph-mgr-dashboard-15.2.5-588.ga7b623 3.5 MB/s | 3.5 MB 00:01 2020-11-20T13:53:26.508 INFO:teuthology.orchestra.run.smithi117.stdout:(48/133): openblas-threads-0.3.3-2.el8.x86_64.r 67 MB/s | 4.4 MB 00:00 2020-11-20T13:53:26.512 INFO:teuthology.orchestra.run.smithi117.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 56 MB/s | 214 kB 00:00 2020-11-20T13:53:26.521 INFO:teuthology.orchestra.run.smithi117.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 64 MB/s | 538 kB 00:00 2020-11-20T13:53:26.573 INFO:teuthology.orchestra.run.smithi117.stdout:(51/133): python3-jsonpatch-1.21-2.el8.noarch.r 21 MB/s | 27 kB 00:00 2020-11-20T13:53:26.573 INFO:teuthology.orchestra.run.smithi117.stdout:(52/133): python3-jsonpointer-1.10-11.el8.noarc 17 MB/s | 20 kB 00:00 2020-11-20T13:53:26.573 INFO:teuthology.orchestra.run.smithi117.stdout:(53/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 58 MB/s | 1.5 MB 00:00 2020-11-20T13:53:26.574 INFO:teuthology.orchestra.run.smithi117.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 39 MB/s | 157 kB 00:00 2020-11-20T13:53:26.574 INFO:teuthology.orchestra.run.smithi117.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 20 MB/s | 39 kB 00:00 2020-11-20T13:53:26.609 INFO:teuthology.orchestra.run.smithi117.stdout:(56/133): python3-babel-2.5.1-5.el8.noarch.rpm 32 MB/s | 4.8 MB 00:00 2020-11-20T13:53:26.615 INFO:teuthology.orchestra.run.smithi117.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 34 MB/s | 225 kB 00:00 2020-11-20T13:53:26.618 INFO:teuthology.orchestra.run.smithi117.stdout:(58/133): python3-prettytable-0.7.2-14.el8.noar 18 MB/s | 44 kB 00:00 2020-11-20T13:53:26.622 INFO:teuthology.orchestra.run.smithi117.stdout:(59/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 31 MB/s | 126 kB 00:00 2020-11-20T13:53:26.626 INFO:teuthology.orchestra.run.smithi117.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 27 MB/s | 110 kB 00:00 2020-11-20T13:53:26.637 INFO:teuthology.orchestra.run.smithi117.stdout:(61/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 46 MB/s | 3.7 MB 00:00 2020-11-20T13:53:26.638 INFO:teuthology.orchestra.run.smithi117.stdout:(62/133): python3-pytz-2017.2-9.el8.noarch.rpm 4.7 MB/s | 54 kB 00:00 2020-11-20T13:53:26.640 INFO:teuthology.orchestra.run.smithi117.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 12 MB/s | 27 kB 00:00 2020-11-20T13:53:26.653 INFO:teuthology.orchestra.run.smithi117.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 37 MB/s | 457 kB 00:00 2020-11-20T13:53:26.660 INFO:teuthology.orchestra.run.smithi117.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 41 MB/s | 298 kB 00:00 2020-11-20T13:53:26.663 INFO:teuthology.orchestra.run.smithi117.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 20 MB/s | 54 kB 00:00 2020-11-20T13:53:26.666 INFO:teuthology.orchestra.run.smithi117.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 24 MB/s | 82 kB 00:00 2020-11-20T13:53:26.668 INFO:teuthology.orchestra.run.smithi117.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 9.9 MB/s | 21 kB 00:00 2020-11-20T13:53:26.671 INFO:teuthology.orchestra.run.smithi117.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 18 MB/s | 48 kB 00:00 2020-11-20T13:53:26.683 INFO:teuthology.orchestra.run.smithi117.stdout:(70/133): krb5-devel-1.17-9.el8.x86_64.rpm 47 MB/s | 548 kB 00:00 2020-11-20T13:53:26.685 INFO:teuthology.orchestra.run.smithi117.stdout:(71/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 8.0 MB/s | 19 kB 00:00 2020-11-20T13:53:26.688 INFO:teuthology.orchestra.run.smithi117.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 15 MB/s | 38 kB 00:00 2020-11-20T13:53:26.700 INFO:teuthology.orchestra.run.smithi085.stdout:(29/133): rbd-mirror-15.2.5-588.ga7b62340.el8.x 8.2 MB/s | 2.7 MB 00:00 2020-11-20T13:53:26.701 INFO:teuthology.orchestra.run.smithi117.stdout:(73/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 50 MB/s | 638 kB 00:00 2020-11-20T13:53:26.706 INFO:teuthology.orchestra.run.smithi117.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 32 MB/s | 169 kB 00:00 2020-11-20T13:53:26.709 INFO:teuthology.orchestra.run.smithi117.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 17 MB/s | 47 kB 00:00 2020-11-20T13:53:26.715 INFO:teuthology.orchestra.run.smithi117.stdout:(76/133): libselinux-devel-2.9-2.1.el8.x86_64.r 33 MB/s | 199 kB 00:00 2020-11-20T13:53:26.719 INFO:teuthology.orchestra.run.smithi117.stdout:(77/133): libsepol-devel-2.9-1.el8.x86_64.rpm 23 MB/s | 86 kB 00:00 2020-11-20T13:53:26.722 INFO:teuthology.orchestra.run.smithi117.stdout:(78/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 28 MB/s | 95 kB 00:00 2020-11-20T13:53:26.724 INFO:teuthology.orchestra.run.smithi117.stdout:(79/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 8.4 MB/s | 18 kB 00:00 2020-11-20T13:53:26.732 INFO:teuthology.orchestra.run.smithi117.stdout:(80/133): libxslt-1.1.32-3.el8.x86_64.rpm 35 MB/s | 249 kB 00:00 2020-11-20T13:53:26.732 INFO:teuthology.orchestra.run.smithi085.stdout:(30/133): rbd-nbd-15.2.5-588.ga7b62340.el8.x86_ 4.3 MB/s | 141 kB 00:00 2020-11-20T13:53:26.755 INFO:teuthology.orchestra.run.smithi085.stdout:(31/133): ceph-grafana-dashboards-15.2.5-588.ga 1.0 MB/s | 24 kB 00:00 2020-11-20T13:53:26.776 INFO:teuthology.orchestra.run.smithi117.stdout:(81/133): ceph-mgr-diskprediction-local-15.2.5- 6.5 MB/s | 8.6 MB 00:01 2020-11-20T13:53:26.779 INFO:teuthology.orchestra.run.smithi085.stdout:(32/133): ceph-mgr-cephadm-15.2.5-588.ga7b62340 2.7 MB/s | 65 kB 00:00 2020-11-20T13:53:26.794 INFO:teuthology.orchestra.run.smithi117.stdout:(82/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 37 MB/s | 2.3 MB 00:00 2020-11-20T13:53:26.800 INFO:teuthology.orchestra.run.smithi117.stdout:(83/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 40 MB/s | 228 kB 00:00 2020-11-20T13:53:26.806 INFO:teuthology.orchestra.run.smithi117.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 38 MB/s | 220 kB 00:00 2020-11-20T13:53:26.812 INFO:teuthology.orchestra.run.smithi117.stdout:(85/133): python3-chardet-3.0.4-7.el8.noarch.rp 36 MB/s | 195 kB 00:00 2020-11-20T13:53:26.814 INFO:teuthology.orchestra.run.smithi117.stdout:(86/133): python3-jwt-1.6.1-2.el8.noarch.rpm 16 MB/s | 43 kB 00:00 2020-11-20T13:53:26.820 INFO:teuthology.orchestra.run.smithi117.stdout:(87/133): python3-oauthlib-2.1.0-1.el8.noarch.r 30 MB/s | 155 kB 00:00 2020-11-20T13:53:26.823 INFO:teuthology.orchestra.run.smithi117.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 11 MB/s | 34 kB 00:00 2020-11-20T13:53:26.828 INFO:teuthology.orchestra.run.smithi117.stdout:(89/133): python3-requests-2.20.0-2.1.el8_1.noa 25 MB/s | 123 kB 00:00 2020-11-20T13:53:26.831 INFO:teuthology.orchestra.run.smithi117.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 14 MB/s | 43 kB 00:00 2020-11-20T13:53:26.837 INFO:teuthology.orchestra.run.smithi117.stdout:(91/133): python3-urllib3-1.24.2-2.el8.noarch.r 31 MB/s | 176 kB 00:00 2020-11-20T13:53:26.841 INFO:teuthology.orchestra.run.smithi117.stdout:(92/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 17 MB/s | 69 kB 00:00 2020-11-20T13:53:26.845 INFO:teuthology.orchestra.run.smithi117.stdout:(93/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 15 MB/s | 56 kB 00:00 2020-11-20T13:53:26.861 INFO:teuthology.orchestra.run.smithi117.stdout:(94/133): pcre2-devel-10.32-1.el8.x86_64.rpm 6.9 MB/s | 605 kB 00:00 2020-11-20T13:53:26.938 INFO:teuthology.orchestra.run.smithi117.stdout:(95/133): python3-scipy-1.0.0-20.module_el8.1.0 48 MB/s | 15 MB 00:00 2020-11-20T13:53:27.046 INFO:teuthology.orchestra.run.smithi117.stdout:(96/133): fmt-6.2.1-1.el8.x86_64.rpm 581 kB/s | 116 kB 00:00 2020-11-20T13:53:27.050 INFO:teuthology.orchestra.run.smithi117.stdout:(97/133): leveldb-1.22-1.el8.x86_64.rpm 1.6 MB/s | 181 kB 00:00 2020-11-20T13:53:27.066 INFO:teuthology.orchestra.run.smithi117.stdout:(98/133): gperftools-libs-2.7-6.el8.x86_64.rpm 1.5 MB/s | 315 kB 00:00 2020-11-20T13:53:27.072 INFO:teuthology.orchestra.run.smithi085.stdout:(33/133): ceph-mgr-dashboard-15.2.5-588.ga7b623 12 MB/s | 3.5 MB 00:00 2020-11-20T13:53:27.093 INFO:teuthology.orchestra.run.smithi117.stdout:(99/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.7 MB/s | 75 kB 00:00 2020-11-20T13:53:27.098 INFO:teuthology.orchestra.run.smithi117.stdout:(100/133): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2020-11-20T13:53:27.102 INFO:teuthology.orchestra.run.smithi117.stdout:(101/133): python3-apipkg-1.5-6.el8.noarch.rpm 593 kB/s | 21 kB 00:00 2020-11-20T13:53:27.113 INFO:teuthology.orchestra.run.smithi117.stdout:(102/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.2 MB/s | 44 kB 00:00 2020-11-20T13:53:27.130 INFO:teuthology.orchestra.run.smithi117.stdout:(103/133): python3-cachetools-3.1.1-4.el8.noarc 1.1 MB/s | 33 kB 00:00 2020-11-20T13:53:27.133 INFO:teuthology.orchestra.run.smithi117.stdout:(104/133): python3-certifi-2018.10.15-7.el8.noa 811 kB/s | 16 kB 00:00 2020-11-20T13:53:27.153 INFO:teuthology.orchestra.run.smithi117.stdout:(105/133): python3-beautifulsoup4-4.6.3-2.el8.1 3.3 MB/s | 185 kB 00:00 2020-11-20T13:53:27.161 INFO:teuthology.orchestra.run.smithi117.stdout:(106/133): python3-cheroot-8.2.1-1.el8.noarch.r 4.6 MB/s | 145 kB 00:00 2020-11-20T13:53:27.179 INFO:teuthology.orchestra.run.smithi085.stdout:(34/133): ceph-osd-15.2.5-588.ga7b62340.el8.x86 2.9 MB/s | 17 MB 00:05 2020-11-20T13:53:27.182 INFO:teuthology.orchestra.run.smithi117.stdout:(107/133): python3-cssselect-0.9.2-10.el8.noarc 1.3 MB/s | 40 kB 00:00 2020-11-20T13:53:27.193 INFO:teuthology.orchestra.run.smithi117.stdout:(108/133): python3-cherrypy-18.4.0-1.el8.noarch 6.3 MB/s | 384 kB 00:00 2020-11-20T13:53:27.212 INFO:teuthology.orchestra.run.smithi117.stdout:(109/133): python3-execnet-1.7.1-1.el8.noarch.r 6.8 MB/s | 355 kB 00:00 2020-11-20T13:53:27.219 INFO:teuthology.orchestra.run.smithi117.stdout:(110/133): python3-google-auth-1.1.1-10.el8.noa 2.2 MB/s | 81 kB 00:00 2020-11-20T13:53:27.222 INFO:teuthology.orchestra.run.smithi085.stdout:(35/133): ceph-mgr-modules-core-15.2.5-588.ga7b 4.1 MB/s | 176 kB 00:00 2020-11-20T13:53:27.224 INFO:teuthology.orchestra.run.smithi117.stdout:(111/133): python3-jaraco-6.2-6.el8.noarch.rpm 366 kB/s | 11 kB 00:00 2020-11-20T13:53:27.236 INFO:teuthology.orchestra.run.smithi117.stdout:(112/133): python3-jaraco-functools-2.0-4.el8.n 742 kB/s | 18 kB 00:00 2020-11-20T13:53:27.245 INFO:teuthology.orchestra.run.smithi085.stdout:(36/133): ceph-mgr-rook-15.2.5-588.ga7b62340.el 1.1 MB/s | 27 kB 00:00 2020-11-20T13:53:27.251 INFO:teuthology.orchestra.run.smithi117.stdout:(113/133): python3-logutils-0.3.5-11.el8.noarch 1.7 MB/s | 49 kB 00:00 2020-11-20T13:53:27.267 INFO:teuthology.orchestra.run.smithi117.stdout:(114/133): python3-more-itertools-7.2.0-3.el8.n 1.9 MB/s | 59 kB 00:00 2020-11-20T13:53:27.268 INFO:teuthology.orchestra.run.smithi085.stdout:(37/133): ceph-prometheus-alerts-15.2.5-588.ga7 380 kB/s | 8.7 kB 00:00 2020-11-20T13:53:27.280 INFO:teuthology.orchestra.run.smithi117.stdout:(115/133): python3-pecan-1.3.2-9.el8.noarch.rpm 9.7 MB/s | 283 kB 00:00 2020-11-20T13:53:27.293 INFO:teuthology.orchestra.run.smithi117.stdout:(116/133): python3-portend-2.6-1.el8.noarch.rpm 604 kB/s | 16 kB 00:00 2020-11-20T13:53:27.300 INFO:teuthology.orchestra.run.smithi117.stdout:(117/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.3 MB/s | 48 kB 00:00 2020-11-20T13:53:27.305 INFO:teuthology.orchestra.run.smithi085.stdout:(38/133): automake-1.16.1-6.el8.noarch.rpm 19 MB/s | 713 kB 00:00 2020-11-20T13:53:27.309 INFO:teuthology.orchestra.run.smithi117.stdout:(118/133): python3-kubernetes-11.0.0-2.el8.noar 13 MB/s | 1.2 MB 00:00 2020-11-20T13:53:27.314 INFO:teuthology.orchestra.run.smithi085.stdout:(39/133): bison-3.0.4-10.el8.x86_64.rpm 79 MB/s | 688 kB 00:00 2020-11-20T13:53:27.319 INFO:teuthology.orchestra.run.smithi085.stdout:(40/133): flex-2.6.1-9.el8.x86_64.rpm 67 MB/s | 320 kB 00:00 2020-11-20T13:53:27.321 INFO:teuthology.orchestra.run.smithi085.stdout:(41/133): jq-1.5-12.el8.x86_64.rpm 61 MB/s | 161 kB 00:00 2020-11-20T13:53:27.322 INFO:teuthology.orchestra.run.smithi117.stdout:(119/133): python3-routes-2.4.1-12.el8.noarch.r 8.9 MB/s | 196 kB 00:00 2020-11-20T13:53:27.324 INFO:teuthology.orchestra.run.smithi117.stdout:(120/133): python3-repoze-lru-0.7-6.el8.noarch. 1.1 MB/s | 34 kB 00:00 2020-11-20T13:53:27.326 INFO:teuthology.orchestra.run.smithi085.stdout:(42/133): librdkafka-0.11.4-1.el8.x86_64.rpm 71 MB/s | 353 kB 00:00 2020-11-20T13:53:27.336 INFO:teuthology.orchestra.run.smithi085.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 73 MB/s | 709 kB 00:00 2020-11-20T13:53:27.340 INFO:teuthology.orchestra.run.smithi085.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 70 MB/s | 259 kB 00:00 2020-11-20T13:53:27.343 INFO:teuthology.orchestra.run.smithi085.stdout:(45/133): oniguruma-6.8.2-1.el8.x86_64.rpm 67 MB/s | 188 kB 00:00 2020-11-20T13:53:27.346 INFO:teuthology.orchestra.run.smithi117.stdout:(121/133): python3-simplegeneric-0.8.1-17.el8.n 775 kB/s | 19 kB 00:00 2020-11-20T13:53:27.351 INFO:teuthology.orchestra.run.smithi117.stdout:(122/133): python3-singledispatch-3.4.0.3-18.el 871 kB/s | 24 kB 00:00 2020-11-20T13:53:27.355 INFO:teuthology.orchestra.run.smithi117.stdout:(123/133): python3-rsa-3.4.2-15.el8.noarch.rpm 1.5 MB/s | 73 kB 00:00 2020-11-20T13:53:27.366 INFO:teuthology.orchestra.run.smithi117.stdout:(124/133): python3-tempora-1.14.1-5.el8.noarch. 1.4 MB/s | 29 kB 00:00 2020-11-20T13:53:27.383 INFO:teuthology.orchestra.run.smithi117.stdout:(125/133): python3-trustme-0.6.0-4.el8.noarch.r 861 kB/s | 27 kB 00:00 2020-11-20T13:53:27.388 INFO:teuthology.orchestra.run.smithi117.stdout:(126/133): python3-waitress-1.2.1-2.el8.1.noarc 7.3 MB/s | 241 kB 00:00 2020-11-20T13:53:27.393 INFO:teuthology.orchestra.run.smithi085.stdout:(46/133): openblas-0.3.3-2.el8.x86_64.rpm 86 MB/s | 4.3 MB 00:00 2020-11-20T13:53:27.395 INFO:teuthology.orchestra.run.smithi117.stdout:(127/133): python3-webob-1.8.5-1.el8.1.noarch.r 8.6 MB/s | 251 kB 00:00 2020-11-20T13:53:27.412 INFO:teuthology.orchestra.run.smithi117.stdout:(128/133): python3-websocket-client-0.56.0-5.el 2.1 MB/s | 61 kB 00:00 2020-11-20T13:53:27.419 INFO:teuthology.orchestra.run.smithi117.stdout:(129/133): python3-webtest-2.0.33-1.el8.noarch. 2.7 MB/s | 85 kB 00:00 2020-11-20T13:53:27.424 INFO:teuthology.orchestra.run.smithi117.stdout:(130/133): python3-zc-lockfile-2.0-2.el8.noarch 799 kB/s | 23 kB 00:00 2020-11-20T13:53:27.443 INFO:teuthology.orchestra.run.smithi117.stdout:(131/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 2.2 MB/s | 69 kB 00:00 2020-11-20T13:53:27.446 INFO:teuthology.orchestra.run.smithi085.stdout:(47/133): openblas-threads-0.3.3-2.el8.x86_64.r 86 MB/s | 4.4 MB 00:00 2020-11-20T13:53:27.501 INFO:teuthology.orchestra.run.smithi085.stdout:(48/133): python3-babel-2.5.1-5.el8.noarch.rpm 86 MB/s | 4.8 MB 00:00 2020-11-20T13:53:27.504 INFO:teuthology.orchestra.run.smithi085.stdout:(49/133): python3-html5lib-0.999999999-6.el8.no 62 MB/s | 214 kB 00:00 2020-11-20T13:53:27.511 INFO:teuthology.orchestra.run.smithi085.stdout:(50/133): python3-jinja2-2.10.1-2.el8_0.noarch. 79 MB/s | 538 kB 00:00 2020-11-20T13:53:27.512 INFO:teuthology.orchestra.run.smithi085.stdout:(51/133): python3-jsonpatch-1.21-2.el8.noarch.r 26 MB/s | 27 kB 00:00 2020-11-20T13:53:27.513 INFO:teuthology.orchestra.run.smithi085.stdout:(52/133): python3-jsonpointer-1.10-11.el8.noarc 19 MB/s | 20 kB 00:00 2020-11-20T13:53:27.531 INFO:teuthology.orchestra.run.smithi085.stdout:(53/133): python3-lxml-4.2.3-1.el8.x86_64.rpm 83 MB/s | 1.5 MB 00:00 2020-11-20T13:53:27.534 INFO:teuthology.orchestra.run.smithi085.stdout:(54/133): python3-mako-1.0.6-13.el8.noarch.rpm 63 MB/s | 157 kB 00:00 2020-11-20T13:53:27.535 INFO:teuthology.orchestra.run.smithi085.stdout:(55/133): python3-markupsafe-0.23-19.el8.x86_64 35 MB/s | 39 kB 00:00 2020-11-20T13:53:27.579 INFO:teuthology.orchestra.run.smithi085.stdout:(56/133): python3-numpy-1.14.3-9.el8.x86_64.rpm 86 MB/s | 3.7 MB 00:00 2020-11-20T13:53:27.583 INFO:teuthology.orchestra.run.smithi085.stdout:(57/133): python3-numpy-f2py-1.14.3-9.el8.x86_6 64 MB/s | 225 kB 00:00 2020-11-20T13:53:27.584 INFO:teuthology.orchestra.run.smithi085.stdout:(58/133): python3-prettytable-0.7.2-14.el8.noar 38 MB/s | 44 kB 00:00 2020-11-20T13:53:27.586 INFO:teuthology.orchestra.run.smithi085.stdout:(59/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 56 MB/s | 126 kB 00:00 2020-11-20T13:53:27.588 INFO:teuthology.orchestra.run.smithi085.stdout:(60/133): python3-pyasn1-modules-0.3.7-6.el8.no 57 MB/s | 110 kB 00:00 2020-11-20T13:53:27.590 INFO:teuthology.orchestra.run.smithi085.stdout:(61/133): python3-pytz-2017.2-9.el8.noarch.rpm 44 MB/s | 54 kB 00:00 2020-11-20T13:53:27.779 INFO:teuthology.orchestra.run.smithi117.stdout:(132/133): librbd1-15.2.5-588.ga7b62340.el8.x86 5.5 MB/s | 1.9 MB 00:00 2020-11-20T13:53:27.803 INFO:teuthology.orchestra.run.smithi085.stdout:(62/133): python3-scipy-1.0.0-20.module_el8.1.0 68 MB/s | 15 MB 00:00 2020-11-20T13:53:27.804 INFO:teuthology.orchestra.run.smithi085.stdout:(63/133): python3-webencodings-0.5.1-6.el8.noar 21 MB/s | 27 kB 00:00 2020-11-20T13:53:27.810 INFO:teuthology.orchestra.run.smithi085.stdout:(64/133): python3-werkzeug-0.12.2-4.el8.noarch. 76 MB/s | 457 kB 00:00 2020-11-20T13:53:27.814 INFO:teuthology.orchestra.run.smithi085.stdout:(65/133): socat-1.7.3.2-6.el8.x86_64.rpm 72 MB/s | 298 kB 00:00 2020-11-20T13:53:27.816 INFO:teuthology.orchestra.run.smithi085.stdout:(66/133): elfutils-libelf-devel-0.176-5.el8.x86 41 MB/s | 54 kB 00:00 2020-11-20T13:53:27.818 INFO:teuthology.orchestra.run.smithi085.stdout:(67/133): fuse-2.9.7-12.el8.x86_64.rpm 49 MB/s | 82 kB 00:00 2020-11-20T13:53:27.819 INFO:teuthology.orchestra.run.smithi085.stdout:(68/133): fuse-common-3.2.1-12.el8.x86_64.rpm 21 MB/s | 21 kB 00:00 2020-11-20T13:53:27.852 INFO:teuthology.orchestra.run.smithi085.stdout:(69/133): keyutils-libs-devel-1.5.10-6.el8.x86_ 1.4 MB/s | 48 kB 00:00 2020-11-20T13:53:27.877 INFO:teuthology.orchestra.run.smithi085.stdout:(70/133): ceph-mgr-diskprediction-local-15.2.5- 11 MB/s | 8.6 MB 00:00 2020-11-20T13:53:27.884 INFO:teuthology.orchestra.run.smithi085.stdout:(71/133): krb5-devel-1.17-9.el8.x86_64.rpm 17 MB/s | 548 kB 00:00 2020-11-20T13:53:27.886 INFO:teuthology.orchestra.run.smithi085.stdout:(72/133): libcom_err-devel-1.44.6-3.el8.x86_64. 25 MB/s | 38 kB 00:00 2020-11-20T13:53:27.903 INFO:teuthology.orchestra.run.smithi085.stdout:(73/133): libgfortran-8.3.1-4.5.el8.x86_64.rpm 38 MB/s | 638 kB 00:00 2020-11-20T13:53:27.905 INFO:teuthology.orchestra.run.smithi085.stdout:(74/133): libquadmath-8.3.1-4.5.el8.x86_64.rpm 65 MB/s | 169 kB 00:00 2020-11-20T13:53:27.907 INFO:teuthology.orchestra.run.smithi085.stdout:(75/133): librabbitmq-0.9.0-1.el8.x86_64.rpm 38 MB/s | 47 kB 00:00 2020-11-20T13:53:27.908 INFO:teuthology.orchestra.run.smithi085.stdout:(76/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 592 kB/s | 19 kB 00:00 2020-11-20T13:53:27.911 INFO:teuthology.orchestra.run.smithi085.stdout:(77/133): libselinux-devel-2.9-2.1.el8.x86_64.r 62 MB/s | 199 kB 00:00 2020-11-20T13:53:27.938 INFO:teuthology.orchestra.run.smithi085.stdout:(78/133): libsepol-devel-2.9-1.el8.x86_64.rpm 2.8 MB/s | 86 kB 00:00 2020-11-20T13:53:27.939 INFO:teuthology.orchestra.run.smithi085.stdout:(79/133): libuuid-devel-2.32.1-17.el8.x86_64.rp 3.2 MB/s | 95 kB 00:00 2020-11-20T13:53:27.940 INFO:teuthology.orchestra.run.smithi085.stdout:(80/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 12 MB/s | 18 kB 00:00 2020-11-20T13:53:27.943 INFO:teuthology.orchestra.run.smithi085.stdout:(81/133): libxslt-1.1.32-3.el8.x86_64.rpm 64 MB/s | 249 kB 00:00 2020-11-20T13:53:27.952 INFO:teuthology.orchestra.run.smithi085.stdout:(82/133): pcre2-devel-10.32-1.el8.x86_64.rpm 67 MB/s | 605 kB 00:00 2020-11-20T13:53:27.957 INFO:teuthology.orchestra.run.smithi085.stdout:(83/133): pcre2-utf16-10.32-1.el8.x86_64.rpm 52 MB/s | 228 kB 00:00 2020-11-20T13:53:27.961 INFO:teuthology.orchestra.run.smithi085.stdout:(84/133): pcre2-utf32-10.32-1.el8.x86_64.rpm 52 MB/s | 220 kB 00:00 2020-11-20T13:53:27.965 INFO:teuthology.orchestra.run.smithi085.stdout:(85/133): python3-chardet-3.0.4-7.el8.noarch.rp 49 MB/s | 195 kB 00:00 2020-11-20T13:53:27.967 INFO:teuthology.orchestra.run.smithi085.stdout:(86/133): python3-jwt-1.6.1-2.el8.noarch.rpm 29 MB/s | 43 kB 00:00 2020-11-20T13:53:27.970 INFO:teuthology.orchestra.run.smithi085.stdout:(87/133): python3-oauthlib-2.1.0-1.el8.noarch.r 43 MB/s | 155 kB 00:00 2020-11-20T13:53:27.972 INFO:teuthology.orchestra.run.smithi085.stdout:(88/133): python3-pysocks-1.6.8-3.el8.noarch.rp 20 MB/s | 34 kB 00:00 2020-11-20T13:53:27.976 INFO:teuthology.orchestra.run.smithi085.stdout:(89/133): python3-requests-2.20.0-2.1.el8_1.noa 37 MB/s | 123 kB 00:00 2020-11-20T13:53:27.978 INFO:teuthology.orchestra.run.smithi085.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 21 MB/s | 43 kB 00:00 2020-11-20T13:53:27.982 INFO:teuthology.orchestra.run.smithi085.stdout:(91/133): python3-urllib3-1.24.2-2.el8.noarch.r 41 MB/s | 176 kB 00:00 2020-11-20T13:53:27.989 INFO:teuthology.orchestra.run.smithi085.stdout:(92/133): openssl-devel-1.1.1c-2.el8_1.1.x86_64 47 MB/s | 2.3 MB 00:00 2020-11-20T13:53:27.990 INFO:teuthology.orchestra.run.smithi085.stdout:(93/133): xfsprogs-devel-5.0.0-1.el8.x86_64.rpm 9.4 MB/s | 69 kB 00:00 2020-11-20T13:53:27.990 INFO:teuthology.orchestra.run.smithi085.stdout:(94/133): zlib-devel-1.2.11-10.el8.x86_64.rpm 36 MB/s | 56 kB 00:00 2020-11-20T13:53:28.003 INFO:teuthology.orchestra.run.smithi117.stdout:(133/133): librados2-15.2.5-588.ga7b62340.el8.x 6.1 MB/s | 3.6 MB 00:00 2020-11-20T13:53:28.007 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2020-11-20T13:53:28.007 INFO:teuthology.orchestra.run.smithi117.stdout:Total 21 MB/s | 185 MB 00:08 2020-11-20T13:53:28.064 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2020-11-20T13:53:28.071 INFO:teuthology.orchestra.run.smithi085.stdout:(95/133): fmt-6.2.1-1.el8.x86_64.rpm 1.4 MB/s | 116 kB 00:00 2020-11-20T13:53:28.087 INFO:teuthology.orchestra.run.smithi085.stdout:(96/133): gperftools-libs-2.7-6.el8.x86_64.rpm 3.2 MB/s | 315 kB 00:00 2020-11-20T13:53:28.104 INFO:teuthology.orchestra.run.smithi085.stdout:(97/133): leveldb-1.22-1.el8.x86_64.rpm 5.4 MB/s | 181 kB 00:00 2020-11-20T13:53:28.105 INFO:teuthology.orchestra.run.smithi085.stdout:(98/133): liboath-2.6.2-3.el8.x86_64.rpm 3.4 MB/s | 59 kB 00:00 2020-11-20T13:53:28.121 INFO:teuthology.orchestra.run.smithi085.stdout:(99/133): libunwind-1.3.1-3.el8.x86_64.rpm 4.4 MB/s | 75 kB 00:00 2020-11-20T13:53:28.121 INFO:teuthology.orchestra.run.smithi085.stdout:(100/133): python3-apipkg-1.5-6.el8.noarch.rpm 1.3 MB/s | 21 kB 00:00 2020-11-20T13:53:28.137 INFO:teuthology.orchestra.run.smithi085.stdout:(101/133): python3-bcrypt-3.1.6-2.el8.1.x86_64. 2.6 MB/s | 44 kB 00:00 2020-11-20T13:53:28.139 INFO:teuthology.orchestra.run.smithi085.stdout:(102/133): python3-beautifulsoup4-4.6.3-2.el8.1 10 MB/s | 185 kB 00:00 2020-11-20T13:53:28.154 INFO:teuthology.orchestra.run.smithi085.stdout:(103/133): python3-cachetools-3.1.1-4.el8.noarc 2.0 MB/s | 33 kB 00:00 2020-11-20T13:53:28.156 INFO:teuthology.orchestra.run.smithi085.stdout:(104/133): python3-certifi-2018.10.15-7.el8.noa 990 kB/s | 16 kB 00:00 2020-11-20T13:53:28.172 INFO:teuthology.orchestra.run.smithi085.stdout:(105/133): python3-cheroot-8.2.1-1.el8.noarch.r 8.0 MB/s | 145 kB 00:00 2020-11-20T13:53:28.189 INFO:teuthology.orchestra.run.smithi085.stdout:(106/133): python3-cherrypy-18.4.0-1.el8.noarch 11 MB/s | 384 kB 00:00 2020-11-20T13:53:28.190 INFO:teuthology.orchestra.run.smithi085.stdout:(107/133): python3-cssselect-0.9.2-10.el8.noarc 2.2 MB/s | 40 kB 00:00 2020-11-20T13:53:28.206 INFO:teuthology.orchestra.run.smithi085.stdout:(108/133): python3-google-auth-1.1.1-10.el8.noa 4.7 MB/s | 81 kB 00:00 2020-11-20T13:53:28.208 INFO:teuthology.orchestra.run.smithi085.stdout:(109/133): python3-execnet-1.7.1-1.el8.noarch.r 18 MB/s | 355 kB 00:00 2020-11-20T13:53:28.223 INFO:teuthology.orchestra.run.smithi085.stdout:(110/133): python3-jaraco-6.2-6.el8.noarch.rpm 683 kB/s | 11 kB 00:00 2020-11-20T13:53:28.225 INFO:teuthology.orchestra.run.smithi085.stdout:(111/133): python3-jaraco-functools-2.0-4.el8.n 1.0 MB/s | 18 kB 00:00 2020-11-20T13:53:28.226 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2020-11-20T13:53:28.227 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2020-11-20T13:53:28.241 INFO:teuthology.orchestra.run.smithi085.stdout:(112/133): python3-logutils-0.3.5-11.el8.noarch 2.9 MB/s | 49 kB 00:00 2020-11-20T13:53:28.258 INFO:teuthology.orchestra.run.smithi085.stdout:(113/133): python3-more-itertools-7.2.0-3.el8.n 3.4 MB/s | 59 kB 00:00 2020-11-20T13:53:28.277 INFO:teuthology.orchestra.run.smithi085.stdout:(114/133): python3-pecan-1.3.2-9.el8.noarch.rpm 15 MB/s | 283 kB 00:00 2020-11-20T13:53:28.290 INFO:teuthology.orchestra.run.smithi085.stdout:(115/133): python3-kubernetes-11.0.0-2.el8.noar 17 MB/s | 1.2 MB 00:00 2020-11-20T13:53:28.293 INFO:teuthology.orchestra.run.smithi085.stdout:(116/133): python3-portend-2.6-1.el8.noarch.rpm 958 kB/s | 16 kB 00:00 2020-11-20T13:53:28.307 INFO:teuthology.orchestra.run.smithi085.stdout:(117/133): python3-remoto-1.1.4-4.el8.noarch.rp 2.8 MB/s | 48 kB 00:00 2020-11-20T13:53:28.310 INFO:teuthology.orchestra.run.smithi085.stdout:(118/133): python3-repoze-lru-0.7-6.el8.noarch. 2.0 MB/s | 34 kB 00:00 2020-11-20T13:53:28.324 INFO:teuthology.orchestra.run.smithi085.stdout:(119/133): python3-routes-2.4.1-12.el8.noarch.r 11 MB/s | 196 kB 00:00 2020-11-20T13:53:28.326 INFO:teuthology.orchestra.run.smithi085.stdout:(120/133): python3-rsa-3.4.2-15.el8.noarch.rpm 4.3 MB/s | 73 kB 00:00 2020-11-20T13:53:28.341 INFO:teuthology.orchestra.run.smithi085.stdout:(121/133): python3-simplegeneric-0.8.1-17.el8.n 1.1 MB/s | 19 kB 00:00 2020-11-20T13:53:28.343 INFO:teuthology.orchestra.run.smithi085.stdout:(122/133): python3-singledispatch-3.4.0.3-18.el 1.4 MB/s | 24 kB 00:00 2020-11-20T13:53:28.357 INFO:teuthology.orchestra.run.smithi085.stdout:(123/133): python3-tempora-1.14.1-5.el8.noarch. 1.7 MB/s | 29 kB 00:00 2020-11-20T13:53:28.360 INFO:teuthology.orchestra.run.smithi085.stdout:(124/133): python3-trustme-0.6.0-4.el8.noarch.r 1.6 MB/s | 27 kB 00:00 2020-11-20T13:53:28.377 INFO:teuthology.orchestra.run.smithi085.stdout:(125/133): python3-waitress-1.2.1-2.el8.1.noarc 12 MB/s | 241 kB 00:00 2020-11-20T13:53:28.379 INFO:teuthology.orchestra.run.smithi085.stdout:(126/133): python3-webob-1.8.5-1.el8.1.noarch.r 13 MB/s | 251 kB 00:00 2020-11-20T13:53:28.393 INFO:teuthology.orchestra.run.smithi085.stdout:(127/133): python3-websocket-client-0.56.0-5.el 3.6 MB/s | 61 kB 00:00 2020-11-20T13:53:28.395 INFO:teuthology.orchestra.run.smithi085.stdout:(128/133): python3-webtest-2.0.33-1.el8.noarch. 4.9 MB/s | 85 kB 00:00 2020-11-20T13:53:28.410 INFO:teuthology.orchestra.run.smithi085.stdout:(129/133): python3-zc-lockfile-2.0-2.el8.noarch 1.4 MB/s | 23 kB 00:00 2020-11-20T13:53:28.412 INFO:teuthology.orchestra.run.smithi085.stdout:(130/133): xmlstarlet-1.6.1-11.el8.x86_64.rpm 4.0 MB/s | 69 kB 00:00 2020-11-20T13:53:28.600 INFO:teuthology.orchestra.run.smithi085.stdout:(131/133): librbd1-15.2.5-588.ga7b62340.el8.x86 10 MB/s | 1.9 MB 00:00 2020-11-20T13:53:28.938 INFO:teuthology.orchestra.run.smithi085.stdout:(132/133): librados2-15.2.5-588.ga7b62340.el8.x 6.7 MB/s | 3.6 MB 00:00 2020-11-20T13:53:29.632 INFO:teuthology.orchestra.run.smithi085.stdout:(133/133): ceph-test-15.2.5-588.ga7b62340.el8.x 7.5 MB/s | 42 MB 00:05 2020-11-20T13:53:29.692 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2020-11-20T13:53:29.692 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2020-11-20T13:53:29.696 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2020-11-20T13:53:29.697 INFO:teuthology.orchestra.run.smithi085.stdout:Total 18 MB/s | 185 MB 00:10 2020-11-20T13:53:29.757 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2020-11-20T13:53:29.918 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2020-11-20T13:53:29.918 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2020-11-20T13:53:31.370 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2020-11-20T13:53:31.370 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2020-11-20T13:53:34.884 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2020-11-20T13:53:35.148 INFO:teuthology.orchestra.run.smithi117.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:53:35.225 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:53:35.388 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-20T13:53:35.519 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-20T13:53:35.734 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-20T13:53:35.893 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:53:35.956 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:53:36.123 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-20T13:53:36.343 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:53:36.502 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2020-11-20T13:53:36.718 INFO:teuthology.orchestra.run.smithi085.stdout: Downgrading : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:53:36.744 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:53:36.789 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 1/140 2020-11-20T13:53:36.935 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:53:36.988 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : fmt-6.2.1-1.el8.x86_64 2/140 2020-11-20T13:53:37.124 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/140 2020-11-20T13:53:37.151 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:53:37.331 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : leveldb-1.22-1.el8.x86_64 4/140 2020-11-20T13:53:37.385 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:53:37.473 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:53:37.487 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:53:37.552 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 5/140 2020-11-20T13:53:37.643 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:53:37.737 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:53:37.761 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 6/140 2020-11-20T13:53:37.965 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:53:38.194 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:53:38.334 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:53:38.339 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/140 2020-11-20T13:53:38.519 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:53:38.523 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:53:38.739 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 8/140 2020-11-20T13:53:38.772 INFO:teuthology.orchestra.run.smithi117.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-20T13:53:38.979 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:53:38.981 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-20T13:53:39.068 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 9/140 2020-11-20T13:53:39.203 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-20T13:53:39.215 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:53:39.309 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 10/140 2020-11-20T13:53:39.376 INFO:teuthology.orchestra.run.smithi117.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-20T13:53:39.526 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:53:39.627 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:53:39.775 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:53:39.807 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-20T13:53:39.923 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:53:39.969 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-20T13:53:40.122 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:53:40.133 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-20T13:53:40.334 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-20T13:53:40.369 INFO:teuthology.orchestra.run.smithi085.stdout: Downgrading : krb5-libs-1.17-9.el8.x86_64 13/140 2020-11-20T13:53:40.492 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:53:40.567 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/140 2020-11-20T13:53:40.633 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:53:40.774 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/140 2020-11-20T13:53:40.792 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:53:40.940 INFO:teuthology.orchestra.run.smithi085.stdout: Downgrading : libkadm5-1.17-9.el8.x86_64 16/140 2020-11-20T13:53:41.090 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:53:41.196 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 17/140 2020-11-20T13:53:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:53:41.429 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/140 2020-11-20T13:53:41.492 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:53:41.549 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : zlib-devel-1.2.11-10.el8.x86_64 19/140 2020-11-20T13:53:41.714 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 20/140 2020-11-20T13:53:41.920 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 21/140 2020-11-20T13:53:42.014 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:53:42.114 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:53:42.167 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:53:42.251 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 22/140 2020-11-20T13:53:42.439 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:53:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:53:42.978 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 23/140 2020-11-20T13:53:43.123 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:53:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-20T13:53:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-20T13:53:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-20T13:53:43.669 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 24/140 2020-11-20T13:53:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-20T13:53:43.831 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:53:43.971 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:53:44.084 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 31/140 2020-11-20T13:53:44.320 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:53:44.526 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 25/140 2020-11-20T13:53:44.822 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 26/140 2020-11-20T13:53:45.011 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/140 2020-11-20T13:53:45.201 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/140 2020-11-20T13:53:45.358 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-20T13:53:45.377 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/140 2020-11-20T13:53:45.568 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:53:45.682 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-20T13:53:45.706 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 31/140 2020-11-20T13:53:45.925 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:53:45.962 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-20T13:53:46.141 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:53:46.833 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/140 2020-11-20T13:53:47.144 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/140 2020-11-20T13:53:47.400 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/140 2020-11-20T13:53:47.613 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:53:48.093 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:53:48.401 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-20T13:53:48.628 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-20T13:53:48.852 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-20T13:53:49.088 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-20T13:53:49.277 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-20T13:53:49.439 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 42/140 2020-11-20T13:53:49.561 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 36/140 2020-11-20T13:53:49.891 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 37/140 2020-11-20T13:53:50.055 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 43/140 2020-11-20T13:53:50.124 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 38/140 2020-11-20T13:53:50.292 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 44/140 2020-11-20T13:53:50.374 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 39/140 2020-11-20T13:53:50.440 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-20T13:53:50.620 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-20T13:53:50.652 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/140 2020-11-20T13:53:50.842 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 41/140 2020-11-20T13:53:50.883 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-20T13:53:51.053 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 42/140 2020-11-20T13:53:51.132 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-20T13:53:51.218 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 43/140 2020-11-20T13:53:51.323 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-20T13:53:51.432 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 44/140 2020-11-20T13:53:51.486 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-20T13:53:51.571 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 45/140 2020-11-20T13:53:51.635 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-20T13:53:51.782 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 46/140 2020-11-20T13:53:51.850 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-20T13:53:52.058 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 47/140 2020-11-20T13:53:52.076 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-20T13:53:52.218 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-20T13:53:52.270 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 48/140 2020-11-20T13:53:52.362 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-20T13:53:52.461 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 49/140 2020-11-20T13:53:52.521 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-20T13:53:52.633 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 50/140 2020-11-20T13:53:52.726 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-20T13:53:52.806 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 51/140 2020-11-20T13:53:52.904 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-20T13:53:53.003 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 52/140 2020-11-20T13:53:53.136 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-20T13:53:53.197 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 53/140 2020-11-20T13:53:53.309 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-20T13:53:53.399 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 54/140 2020-11-20T13:53:53.489 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-20T13:53:53.543 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 55/140 2020-11-20T13:53:53.672 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-20T13:53:53.749 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 56/140 2020-11-20T13:53:53.889 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-20T13:53:53.923 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 57/140 2020-11-20T13:53:54.058 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-20T13:53:54.135 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 58/140 2020-11-20T13:53:54.280 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-20T13:53:54.345 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-portend-2.6-1.el8.noarch 59/140 2020-11-20T13:53:54.538 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-20T13:53:54.548 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 60/140 2020-11-20T13:53:54.702 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 61/140 2020-11-20T13:53:54.745 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-20T13:53:54.959 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-20T13:53:55.171 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/140 2020-11-20T13:53:55.439 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-20T13:53:55.450 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/140 2020-11-20T13:53:55.664 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/140 2020-11-20T13:53:55.764 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-20T13:53:55.853 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 65/140 2020-11-20T13:53:55.977 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-20T13:53:56.110 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 66/140 2020-11-20T13:53:56.216 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-20T13:53:56.317 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 67/140 2020-11-20T13:53:56.403 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-20T13:53:56.481 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 68/140 2020-11-20T13:53:56.569 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-20T13:53:56.707 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/140 2020-11-20T13:53:56.887 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 70/140 2020-11-20T13:53:57.099 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 71/140 2020-11-20T13:53:57.321 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-urllib3-1.24.2-2.el8.noarch 72/140 2020-11-20T13:53:57.518 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 73/140 2020-11-20T13:53:57.610 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-20T13:53:57.718 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 74/140 2020-11-20T13:53:57.850 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-20T13:53:58.076 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-20T13:53:58.376 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-20T13:53:58.532 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-20T13:53:58.695 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-20T13:53:58.740 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 75/140 2020-11-20T13:53:58.826 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-20T13:53:58.973 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 76/140 2020-11-20T13:53:59.128 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-20T13:53:59.207 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-utf32-10.32-1.el8.x86_64 77/140 2020-11-20T13:53:59.367 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:53:59.507 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-utf16-10.32-1.el8.x86_64 78/140 2020-11-20T13:53:59.542 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-20T13:53:59.689 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-devel-10.32-1.el8.x86_64 79/140 2020-11-20T13:53:59.770 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-20T13:53:59.852 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 80/140 2020-11-20T13:53:59.974 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-20T13:53:59.974 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-20T13:53:59.974 INFO:teuthology.orchestra.run.smithi117.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-20T13:53:59.975 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:53:59.991 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libuuid-devel-2.32.1-17.el8.x86_64 81/140 2020-11-20T13:54:00.290 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libsepol-devel-2.9-1.el8.x86_64 82/140 2020-11-20T13:54:00.332 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-20T13:54:00.540 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:54:00.583 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-20T13:54:00.665 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcom_err-devel-1.44.6-3.el8.x86_64 84/140 2020-11-20T13:54:00.817 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-20T13:54:00.885 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 85/140 2020-11-20T13:54:01.122 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : krb5-devel-1.17-9.el8.x86_64 86/140 2020-11-20T13:54:01.122 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 87/140 2020-11-20T13:54:01.123 INFO:teuthology.orchestra.run.smithi085.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-11-20T13:54:01.123 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:01.198 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-20T13:54:01.492 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-20T13:54:01.530 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : fuse-2.9.7-12.el8.x86_64 88/140 2020-11-20T13:54:01.772 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 89/140 2020-11-20T13:54:01.837 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-20T13:54:01.990 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 90/140 2020-11-20T13:54:02.162 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-20T13:54:02.313 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-20T13:54:02.366 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 91/140 2020-11-20T13:54:02.655 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 92/140 2020-11-20T13:54:03.001 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 93/140 2020-11-20T13:54:03.295 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 94/140 2020-11-20T13:54:03.470 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 95/140 2020-11-20T13:54:06.096 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:06.345 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:07.022 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:07.148 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:07.350 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:07.386 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:07.440 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:07.594 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:54:08.051 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 96/140 2020-11-20T13:54:08.381 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:08.469 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 97/140 2020-11-20T13:54:08.619 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:54:24.361 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:54:24.361 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2020-11-20T13:54:24.361 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2020-11-20T13:54:24.362 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /dev 2020-11-20T13:54:24.362 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /run 2020-11-20T13:54:24.362 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2020-11-20T13:54:24.362 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2020-11-20T13:54:24.362 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2020-11-20T13:54:24.363 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2020-11-20T13:54:24.363 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /dev 2020-11-20T13:54:24.363 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:24.700 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:54:25.087 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:54:25.279 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:54:25.536 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 98/140 2020-11-20T13:54:25.536 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /sys 2020-11-20T13:54:25.536 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /proc 2020-11-20T13:54:25.537 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2020-11-20T13:54:25.537 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /run 2020-11-20T13:54:25.537 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /mnt 2020-11-20T13:54:25.537 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /var/tmp 2020-11-20T13:54:25.537 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /home 2020-11-20T13:54:25.538 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /tmp 2020-11-20T13:54:25.538 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2020-11-20T13:54:25.538 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:25.908 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:54:26.304 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 99/140 2020-11-20T13:54:26.494 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:54:26.698 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:54:26.912 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:54:26.943 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:54:27.139 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-20T13:54:27.360 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-20T13:54:27.499 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:54:27.590 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:54:27.827 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-20T13:54:27.907 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 100/140 2020-11-20T13:54:28.094 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:54:28.126 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 101/140 2020-11-20T13:54:28.983 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-20T13:54:29.022 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 102/140 2020-11-20T13:54:29.098 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 107/140 2020-11-20T13:54:29.290 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 108/140 2020-11-20T13:54:29.495 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 103/140 2020-11-20T13:54:29.641 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:54:29.728 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 104/140 2020-11-20T13:54:30.016 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : jq-1.5-12.el8.x86_64 105/140 2020-11-20T13:54:30.233 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : automake-1.16.1-6.el8.noarch 106/140 2020-11-20T13:54:30.339 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 107/140 2020-11-20T13:54:30.524 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 108/140 2020-11-20T13:54:30.734 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:54:30.937 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:54:31.106 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:54:31.956 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:54:31.973 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:54:32.144 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 109/140 2020-11-20T13:54:32.261 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:54:32.357 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:54:33.223 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 110/140 2020-11-20T13:54:33.301 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:54:33.485 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:54:33.505 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:54:33.577 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:54:33.883 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 113/140 2020-11-20T13:54:34.529 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 111/140 2020-11-20T13:54:34.562 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:54:34.594 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:54:34.739 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:54:34.813 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 112/140 2020-11-20T13:54:34.972 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:54:34.988 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:54:35.124 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 113/140 2020-11-20T13:54:35.203 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 116/140 2020-11-20T13:54:35.318 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:54:35.345 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 114/140 2020-11-20T13:54:35.363 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:54:35.473 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:54:35.487 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 115/140 2020-11-20T13:54:35.693 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 116/140 2020-11-20T13:54:35.831 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:54:39.255 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:54:40.236 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/140 2020-11-20T13:54:40.476 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 118/140 2020-11-20T13:54:41.154 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:54:41.514 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:54:41.662 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:54:41.732 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:54:41.759 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 118/140 2020-11-20T13:54:41.997 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:54:42.309 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 119/140 2020-11-20T13:54:42.464 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:54:42.534 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 120/140 2020-11-20T13:54:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 121/140 2020-11-20T13:54:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-20T13:54:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-20T13:54:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 124/140 2020-11-20T13:54:43.380 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 121/140 2020-11-20T13:54:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-20T13:54:43.633 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 122/140 2020-11-20T13:54:43.766 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : xfsprogs-devel-5.0.0-1.el8.x86_64 123/140 2020-11-20T13:54:43.944 INFO:teuthology.orchestra.run.smithi117.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-20T13:54:43.949 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 124/140 2020-11-20T13:54:44.158 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 127/140 2020-11-20T13:54:44.292 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : elfutils-libelf-devel-0.176-5.el8.x86_64 125/140 2020-11-20T13:54:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 128/140 2020-11-20T13:54:44.562 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:54:44.666 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:54:44.712 INFO:teuthology.orchestra.run.smithi085.stdout: Downgrading : krb5-workstation-1.17-9.el8.x86_64 126/140 2020-11-20T13:54:44.935 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 127/140 2020-11-20T13:54:44.937 INFO:teuthology.orchestra.run.smithi117.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-20T13:54:45.129 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-20T13:54:45.141 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 128/140 2020-11-20T13:54:45.286 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:54:45.364 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:54:45.377 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:54:45.474 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.5-588.ga7b623 129/140 2020-11-20T13:54:45.596 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:54:45.614 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:54:45.714 INFO:teuthology.orchestra.run.smithi085.stdout: Downgrading : openssl-1:1.1.1c-2.el8_1.1.x86_64 130/140 2020-11-20T13:54:45.788 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-20T13:54:45.906 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/140 2020-11-20T13:54:45.915 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-20T13:54:46.099 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-20T13:54:46.121 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:54:46.207 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/140 2020-11-20T13:54:46.279 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-20T13:54:46.280 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:54:46.438 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:54:46.458 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/140 2020-11-20T13:54:46.531 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:54:46.531 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:54:46.639 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : krb5-workstation-1.17-18.el8.x86_64 134/140 2020-11-20T13:54:46.780 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:54:46.780 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:54:46.792 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : libkadm5-1.17-18.el8.x86_64 135/140 2020-11-20T13:54:46.958 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : openssl-1:1.1.1c-15.el8.x86_64 136/140 2020-11-20T13:54:47.156 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : krb5-libs-1.17-18.el8.x86_64 137/140 2020-11-20T13:54:47.156 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:54:47.381 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:54:47.381 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:54:47.587 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 139/140 2020-11-20T13:54:47.588 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:54:47.624 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:54:47.624 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-20T13:54:47.624 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-20T13:54:47.624 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-20T13:54:47.625 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-20T13:54:47.625 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-20T13:54:47.625 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-20T13:54:47.625 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-20T13:54:47.625 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-20T13:54:47.626 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-20T13:54:47.626 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-20T13:54:47.626 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:54:47.626 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:54:47.627 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 13/140 2020-11-20T13:54:47.627 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 14/140 2020-11-20T13:54:47.627 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 15/140 2020-11-20T13:54:47.627 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 16/140 2020-11-20T13:54:47.627 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 17/140 2020-11-20T13:54:47.628 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 18/140 2020-11-20T13:54:47.628 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 19/140 2020-11-20T13:54:47.628 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 20/140 2020-11-20T13:54:47.628 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 21/140 2020-11-20T13:54:47.628 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 22/140 2020-11-20T13:54:47.629 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 23/140 2020-11-20T13:54:47.629 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 24/140 2020-11-20T13:54:47.629 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 25/140 2020-11-20T13:54:47.629 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 26/140 2020-11-20T13:54:47.630 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 27/140 2020-11-20T13:54:47.630 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 28/140 2020-11-20T13:54:47.630 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 29/140 2020-11-20T13:54:47.630 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:54:47.630 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 31/140 2020-11-20T13:54:47.631 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:54:47.631 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 33/140 2020-11-20T13:54:47.631 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 34/140 2020-11-20T13:54:47.631 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 35/140 2020-11-20T13:54:47.631 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 36/140 2020-11-20T13:54:47.632 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 37/140 2020-11-20T13:54:47.632 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 38/140 2020-11-20T13:54:47.632 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 39/140 2020-11-20T13:54:47.632 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 40/140 2020-11-20T13:54:47.632 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 41/140 2020-11-20T13:54:47.633 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 42/140 2020-11-20T13:54:47.633 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 43/140 2020-11-20T13:54:47.633 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 44/140 2020-11-20T13:54:47.633 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-20T13:54:47.633 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-20T13:54:47.634 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-20T13:54:47.634 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-20T13:54:47.634 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-20T13:54:47.634 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-20T13:54:47.634 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-20T13:54:47.635 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-20T13:54:47.635 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-20T13:54:47.635 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-20T13:54:47.635 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-20T13:54:47.635 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-20T13:54:47.635 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-20T13:54:47.636 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-20T13:54:47.636 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-20T13:54:47.636 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-20T13:54:47.636 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-20T13:54:47.637 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-20T13:54:47.637 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-20T13:54:47.637 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-20T13:54:47.637 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-20T13:54:47.637 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-20T13:54:47.638 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-20T13:54:47.638 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-20T13:54:47.638 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-20T13:54:47.638 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-20T13:54:47.638 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-20T13:54:47.639 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-20T13:54:47.639 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-20T13:54:47.639 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-20T13:54:47.639 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-20T13:54:47.639 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-20T13:54:47.640 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-20T13:54:47.640 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-20T13:54:47.640 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-20T13:54:47.640 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-20T13:54:47.640 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-20T13:54:47.641 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-20T13:54:47.641 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:54:47.641 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-20T13:54:47.641 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-20T13:54:47.641 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-20T13:54:47.642 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-20T13:54:47.642 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-20T13:54:47.642 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-20T13:54:47.642 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-20T13:54:47.642 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-20T13:54:47.643 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-20T13:54:47.643 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-20T13:54:47.643 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-20T13:54:47.643 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-20T13:54:47.644 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-20T13:54:47.644 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-20T13:54:47.644 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-20T13:54:47.644 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-20T13:54:47.645 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-20T13:54:47.645 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-20T13:54:47.645 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-20T13:54:47.645 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-20T13:54:47.646 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-20T13:54:47.646 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-20T13:54:47.646 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-20T13:54:47.646 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-20T13:54:47.646 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-20T13:54:47.647 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-20T13:54:47.647 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-20T13:54:47.647 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-20T13:54:47.647 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-20T13:54:47.647 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-20T13:54:47.648 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-20T13:54:47.648 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-20T13:54:47.648 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-20T13:54:47.648 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-20T13:54:47.648 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-20T13:54:47.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-20T13:54:47.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-20T13:54:47.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-20T13:54:47.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-20T13:54:47.649 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-20T13:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-20T13:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-20T13:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-20T13:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-20T13:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-20T13:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-20T13:54:47.651 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-20T13:54:47.651 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-20T13:54:47.651 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-20T13:54:47.651 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-20T13:54:47.651 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-20T13:54:47.652 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-20T13:54:47.652 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-20T13:54:47.652 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 137/140 2020-11-20T13:54:47.652 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:54:47.652 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 139/140 2020-11-20T13:54:48.486 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openssl-libs-1:1.1.1c-15.el8.x86_64 140/140 2020-11-20T13:54:48.486 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : krb5-libs-1.17-9.el8.x86_64 1/140 2020-11-20T13:54:48.486 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : krb5-libs-1.17-18.el8.x86_64 2/140 2020-11-20T13:54:48.487 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : krb5-workstation-1.17-9.el8.x86_64 3/140 2020-11-20T13:54:48.487 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : krb5-workstation-1.17-18.el8.x86_64 4/140 2020-11-20T13:54:48.487 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libkadm5-1.17-9.el8.x86_64 5/140 2020-11-20T13:54:48.487 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libkadm5-1.17-18.el8.x86_64 6/140 2020-11-20T13:54:48.487 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-1:1.1.1c-2.el8_1.1.x86_64 7/140 2020-11-20T13:54:48.488 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-1:1.1.1c-15.el8.x86_64 8/140 2020-11-20T13:54:48.488 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 9/140 2020-11-20T13:54:48.488 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-libs-1:1.1.1c-15.el8.x86_64 10/140 2020-11-20T13:54:48.488 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-2:15.2.5-588.ga7b62340.el8.x86_64 11/140 2020-11-20T13:54:48.488 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 12/140 2020-11-20T13:54:48.489 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 13/140 2020-11-20T13:54:48.489 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 14/140 2020-11-20T13:54:48.489 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-immutable-object-cache-2:15.2.5-588.ga7b623 15/140 2020-11-20T13:54:48.489 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 16/140 2020-11-20T13:54:48.489 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 17/140 2020-11-20T13:54:48.490 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 18/140 2020-11-20T13:54:48.490 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 19/140 2020-11-20T13:54:48.490 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 20/140 2020-11-20T13:54:48.490 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 21/140 2020-11-20T13:54:48.490 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 22/140 2020-11-20T13:54:48.491 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 23/140 2020-11-20T13:54:48.491 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_6 24/140 2020-11-20T13:54:48.491 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 25/140 2020-11-20T13:54:48.491 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 26/140 2020-11-20T13:54:48.492 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_ 27/140 2020-11-20T13:54:48.492 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 28/140 2020-11-20T13:54:48.492 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8 29/140 2020-11-20T13:54:48.492 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x 30/140 2020-11-20T13:54:48.492 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 31/140 2020-11-20T13:54:48.493 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 32/140 2020-11-20T13:54:48.500 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 33/140 2020-11-20T13:54:48.501 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 34/140 2020-11-20T13:54:48.501 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 35/140 2020-11-20T13:54:48.501 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 36/140 2020-11-20T13:54:48.501 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 37/140 2020-11-20T13:54:48.502 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.e 38/140 2020-11-20T13:54:48.502 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noar 39/140 2020-11-20T13:54:48.502 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.no 40/140 2020-11-20T13:54:48.502 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b6 41/140 2020-11-20T13:54:48.503 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8 42/140 2020-11-20T13:54:48.503 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 43/140 2020-11-20T13:54:48.503 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el 44/140 2020-11-20T13:54:48.503 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : automake-1.16.1-6.el8.noarch 45/140 2020-11-20T13:54:48.503 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : bison-3.0.4-10.el8.x86_64 46/140 2020-11-20T13:54:48.504 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : flex-2.6.1-9.el8.x86_64 47/140 2020-11-20T13:54:48.504 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : jq-1.5-12.el8.x86_64 48/140 2020-11-20T13:54:48.504 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 49/140 2020-11-20T13:54:48.504 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 50/140 2020-11-20T13:54:48.504 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 51/140 2020-11-20T13:54:48.505 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 52/140 2020-11-20T13:54:48.505 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 53/140 2020-11-20T13:54:48.505 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 54/140 2020-11-20T13:54:48.505 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 55/140 2020-11-20T13:54:48.505 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 56/140 2020-11-20T13:54:48.506 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 57/140 2020-11-20T13:54:48.506 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 58/140 2020-11-20T13:54:48.506 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 59/140 2020-11-20T13:54:48.506 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 60/140 2020-11-20T13:54:48.506 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 61/140 2020-11-20T13:54:48.507 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 62/140 2020-11-20T13:54:48.507 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 63/140 2020-11-20T13:54:48.507 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 64/140 2020-11-20T13:54:48.507 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 65/140 2020-11-20T13:54:48.507 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 66/140 2020-11-20T13:54:48.508 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 67/140 2020-11-20T13:54:48.508 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 68/140 2020-11-20T13:54:48.508 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.1.0+245+c39af4 69/140 2020-11-20T13:54:48.508 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 70/140 2020-11-20T13:54:48.508 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 71/140 2020-11-20T13:54:48.509 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 72/140 2020-11-20T13:54:48.509 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : elfutils-libelf-devel-0.176-5.el8.x86_64 73/140 2020-11-20T13:54:48.509 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 74/140 2020-11-20T13:54:48.509 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 75/140 2020-11-20T13:54:48.509 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/140 2020-11-20T13:54:48.510 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : krb5-devel-1.17-9.el8.x86_64 77/140 2020-11-20T13:54:48.510 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/140 2020-11-20T13:54:48.510 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcom_err-devel-1.44.6-3.el8.x86_64 79/140 2020-11-20T13:54:48.510 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 80/140 2020-11-20T13:54:48.510 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 81/140 2020-11-20T13:54:48.511 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 82/140 2020-11-20T13:54:48.511 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libselinux-devel-2.9-2.1.el8.x86_64 83/140 2020-11-20T13:54:48.511 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libsepol-devel-2.9-1.el8.x86_64 84/140 2020-11-20T13:54:48.511 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libuuid-devel-2.32.1-17.el8.x86_64 85/140 2020-11-20T13:54:48.511 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/140 2020-11-20T13:54:48.511 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 87/140 2020-11-20T13:54:48.512 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 88/140 2020-11-20T13:54:48.512 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-devel-10.32-1.el8.x86_64 89/140 2020-11-20T13:54:48.512 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-utf16-10.32-1.el8.x86_64 90/140 2020-11-20T13:54:48.512 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-utf32-10.32-1.el8.x86_64 91/140 2020-11-20T13:54:48.512 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 92/140 2020-11-20T13:54:48.513 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 93/140 2020-11-20T13:54:48.513 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/140 2020-11-20T13:54:48.513 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 95/140 2020-11-20T13:54:48.513 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 96/140 2020-11-20T13:54:48.513 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 97/140 2020-11-20T13:54:48.514 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-urllib3-1.24.2-2.el8.noarch 98/140 2020-11-20T13:54:48.514 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : xfsprogs-devel-5.0.0-1.el8.x86_64 99/140 2020-11-20T13:54:48.514 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : zlib-devel-1.2.11-10.el8.x86_64 100/140 2020-11-20T13:54:48.514 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 101/140 2020-11-20T13:54:48.514 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 102/140 2020-11-20T13:54:48.515 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : leveldb-1.22-1.el8.x86_64 103/140 2020-11-20T13:54:48.515 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 104/140 2020-11-20T13:54:48.515 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 105/140 2020-11-20T13:54:48.515 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 106/140 2020-11-20T13:54:48.515 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 107/140 2020-11-20T13:54:48.516 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 108/140 2020-11-20T13:54:48.516 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 109/140 2020-11-20T13:54:48.516 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 110/140 2020-11-20T13:54:48.516 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 111/140 2020-11-20T13:54:48.516 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 112/140 2020-11-20T13:54:48.517 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 113/140 2020-11-20T13:54:48.517 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 114/140 2020-11-20T13:54:48.517 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 115/140 2020-11-20T13:54:48.517 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 116/140 2020-11-20T13:54:48.517 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 117/140 2020-11-20T13:54:48.518 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 118/140 2020-11-20T13:54:48.518 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 119/140 2020-11-20T13:54:48.518 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 120/140 2020-11-20T13:54:48.518 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 121/140 2020-11-20T13:54:48.518 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-portend-2.6-1.el8.noarch 122/140 2020-11-20T13:54:48.519 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 123/140 2020-11-20T13:54:48.519 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 124/140 2020-11-20T13:54:48.519 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 125/140 2020-11-20T13:54:48.519 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 126/140 2020-11-20T13:54:48.519 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 127/140 2020-11-20T13:54:48.519 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 128/140 2020-11-20T13:54:48.520 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 129/140 2020-11-20T13:54:48.520 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 130/140 2020-11-20T13:54:48.520 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 131/140 2020-11-20T13:54:48.520 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 132/140 2020-11-20T13:54:48.520 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 133/140 2020-11-20T13:54:48.521 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 134/140 2020-11-20T13:54:48.521 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 135/140 2020-11-20T13:54:48.521 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 136/140 2020-11-20T13:54:48.521 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-2:15.2.5-588.ga7b62340.el8.x86_64 137/140 2020-11-20T13:54:48.521 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 138/140 2020-11-20T13:54:48.522 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 139/140 2020-11-20T13:54:49.330 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-20T13:54:49.330 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:49.330 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2020-11-20T13:54:49.330 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.331 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.331 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:49.331 INFO:teuthology.orchestra.run.smithi117.stdout:Downgraded: 2020-11-20T13:54:49.331 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-20T13:54:49.332 INFO:teuthology.orchestra.run.smithi117.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:54:49.332 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:54:49.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:49.332 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2020-11-20T13:54:49.332 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.333 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.333 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.333 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.333 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.334 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.334 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.334 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.334 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.334 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.335 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.335 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.335 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.335 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.335 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.336 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.336 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.336 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.336 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:49.336 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:49.337 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:49.337 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:49.337 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.0.4-10.el8.x86_64 2020-11-20T13:54:49.337 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.1-9.el8.x86_64 2020-11-20T13:54:49.337 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-20T13:54:49.338 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-20T13:54:49.338 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-20T13:54:49.338 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-20T13:54:49.338 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:54:49.338 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-20T13:54:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-20T13:54:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-20T13:54:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.341 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:49.341 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:49.341 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:49.341 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:49.341 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.1-6.el8.noarch 2020-11-20T13:54:49.342 INFO:teuthology.orchestra.run.smithi117.stdout: jq-1.5-12.el8.x86_64 2020-11-20T13:54:49.342 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-20T13:54:49.342 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-20T13:54:49.342 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-20T13:54:49.342 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-20T13:54:49.343 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-20T13:54:49.343 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-20T13:54:49.343 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-20T13:54:49.343 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-20T13:54:49.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-20T13:54:49.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-20T13:54:49.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-20T13:54:49.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-20T13:54:49.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-20T13:54:49.344 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-20T13:54:49.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-20T13:54:49.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-20T13:54:49.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-20T13:54:49.345 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-20T13:54:49.346 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-20T13:54:49.346 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-20T13:54:49.346 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-20T13:54:49.346 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-20T13:54:49.346 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-20T13:54:49.346 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-20T13:54:49.347 INFO:teuthology.orchestra.run.smithi117.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-20T13:54:49.347 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-20T13:54:49.347 INFO:teuthology.orchestra.run.smithi117.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-20T13:54:49.347 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-20T13:54:49.347 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-20T13:54:49.348 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-20T13:54:49.348 INFO:teuthology.orchestra.run.smithi117.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-20T13:54:49.348 INFO:teuthology.orchestra.run.smithi117.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-20T13:54:49.348 INFO:teuthology.orchestra.run.smithi117.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-20T13:54:49.348 INFO:teuthology.orchestra.run.smithi117.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-20T13:54:49.349 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-20T13:54:49.349 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-20T13:54:49.349 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-20T13:54:49.349 INFO:teuthology.orchestra.run.smithi117.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-20T13:54:49.350 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-20T13:54:49.350 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-20T13:54:49.350 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-20T13:54:49.350 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-20T13:54:49.350 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-20T13:54:49.351 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-20T13:54:49.351 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-20T13:54:49.351 INFO:teuthology.orchestra.run.smithi117.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-20T13:54:49.351 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-20T13:54:49.351 INFO:teuthology.orchestra.run.smithi117.stdout: leveldb-1.22-1.el8.x86_64 2020-11-20T13:54:49.352 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-20T13:54:49.352 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-20T13:54:49.352 INFO:teuthology.orchestra.run.smithi117.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-20T13:54:49.352 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-20T13:54:49.352 INFO:teuthology.orchestra.run.smithi117.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-20T13:54:49.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-20T13:54:49.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-20T13:54:49.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-20T13:54:49.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-20T13:54:49.353 INFO:teuthology.orchestra.run.smithi117.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-20T13:54:49.354 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-20T13:54:49.354 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-20T13:54:49.354 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-20T13:54:49.354 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-20T13:54:49.355 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-20T13:54:49.355 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-20T13:54:49.355 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-20T13:54:49.355 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-2.6-1.el8.noarch 2020-11-20T13:54:49.355 INFO:teuthology.orchestra.run.smithi117.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-20T13:54:49.356 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-20T13:54:49.356 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-20T13:54:49.356 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-20T13:54:49.356 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-20T13:54:49.356 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-20T13:54:49.357 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-20T13:54:49.357 INFO:teuthology.orchestra.run.smithi117.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-20T13:54:49.357 INFO:teuthology.orchestra.run.smithi117.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-20T13:54:49.357 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-20T13:54:49.357 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-20T13:54:49.358 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-20T13:54:49.358 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-20T13:54:49.358 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-20T13:54:49.358 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:49.359 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2020-11-20T13:54:49.478 DEBUG:teuthology.parallel:result is None 2020-11-20T13:54:50.184 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 140/140 2020-11-20T13:54:50.185 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:50.185 INFO:teuthology.orchestra.run.smithi085.stdout:Upgraded: 2020-11-20T13:54:50.185 INFO:teuthology.orchestra.run.smithi085.stdout: librados2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.185 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.186 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:50.186 INFO:teuthology.orchestra.run.smithi085.stdout:Downgraded: 2020-11-20T13:54:50.186 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-libs-1.17-9.el8.x86_64 krb5-workstation-1.17-9.el8.x86_64 2020-11-20T13:54:50.186 INFO:teuthology.orchestra.run.smithi085.stdout: libkadm5-1.17-9.el8.x86_64 openssl-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:54:50.187 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-libs-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:54:50.187 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:50.187 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2020-11-20T13:54:50.187 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.187 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.188 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.188 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.188 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.188 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.189 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.189 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.189 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.189 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.189 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.190 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.190 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.190 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.190 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.191 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.191 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.191 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.191 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:50.191 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:50.192 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:50.192 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:50.192 INFO:teuthology.orchestra.run.smithi085.stdout: bison-3.0.4-10.el8.x86_64 2020-11-20T13:54:50.192 INFO:teuthology.orchestra.run.smithi085.stdout: flex-2.6.1-9.el8.x86_64 2020-11-20T13:54:50.192 INFO:teuthology.orchestra.run.smithi085.stdout: libtool-2.4.6-25.el8.x86_64 2020-11-20T13:54:50.193 INFO:teuthology.orchestra.run.smithi085.stdout: elfutils-libelf-devel-0.176-5.el8.x86_64 2020-11-20T13:54:50.193 INFO:teuthology.orchestra.run.smithi085.stdout: libaio-devel-0.3.112-1.el8.x86_64 2020-11-20T13:54:50.193 INFO:teuthology.orchestra.run.smithi085.stdout: libuuid-devel-2.32.1-17.el8.x86_64 2020-11-20T13:54:50.193 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-devel-1:1.1.1c-2.el8_1.1.x86_64 2020-11-20T13:54:50.194 INFO:teuthology.orchestra.run.smithi085.stdout: xfsprogs-devel-5.0.0-1.el8.x86_64 2020-11-20T13:54:50.194 INFO:teuthology.orchestra.run.smithi085.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-11-20T13:54:50.194 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-11-20T13:54:50.194 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.194 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.194 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.195 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.195 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.195 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.195 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.195 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.196 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common-2:15.2.5-588.ga7b62340.el8.x86_64 2020-11-20T13:54:50.196 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:50.196 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:50.196 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts-2:15.2.5-588.ga7b62340.el8.noarch 2020-11-20T13:54:50.196 INFO:teuthology.orchestra.run.smithi085.stdout: automake-1.16.1-6.el8.noarch 2020-11-20T13:54:50.197 INFO:teuthology.orchestra.run.smithi085.stdout: jq-1.5-12.el8.x86_64 2020-11-20T13:54:50.197 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-11-20T13:54:50.197 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-11-20T13:54:50.197 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-11-20T13:54:50.198 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-0.3.3-2.el8.x86_64 2020-11-20T13:54:50.198 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-11-20T13:54:50.198 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel-2.5.1-5.el8.noarch 2020-11-20T13:54:50.198 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-11-20T13:54:50.198 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-11-20T13:54:50.199 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-11-20T13:54:50.199 INFO:teuthology.orchestra.run.smithi085.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-11-20T13:54:50.199 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako-1.0.6-13.el8.noarch 2020-11-20T13:54:50.199 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-11-20T13:54:50.199 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-11-20T13:54:50.200 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-11-20T13:54:50.200 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-11-20T13:54:50.200 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-11-20T13:54:50.200 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-11-20T13:54:50.200 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz-2017.2-9.el8.noarch 2020-11-20T13:54:50.201 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy-1.0.0-20.module_el8.1.0+245+c39af44f.x86_64 2020-11-20T13:54:50.201 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-11-20T13:54:50.201 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-11-20T13:54:50.201 INFO:teuthology.orchestra.run.smithi085.stdout: socat-1.7.3.2-6.el8.x86_64 2020-11-20T13:54:50.201 INFO:teuthology.orchestra.run.smithi085.stdout: fuse-2.9.7-12.el8.x86_64 2020-11-20T13:54:50.202 INFO:teuthology.orchestra.run.smithi085.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-11-20T13:54:50.202 INFO:teuthology.orchestra.run.smithi085.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2020-11-20T13:54:50.202 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-devel-1.17-9.el8.x86_64 2020-11-20T13:54:50.202 INFO:teuthology.orchestra.run.smithi085.stdout: libcom_err-devel-1.44.6-3.el8.x86_64 2020-11-20T13:54:50.202 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-11-20T13:54:50.203 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-11-20T13:54:50.203 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-11-20T13:54:50.203 INFO:teuthology.orchestra.run.smithi085.stdout: libselinux-devel-2.9-2.1.el8.x86_64 2020-11-20T13:54:50.203 INFO:teuthology.orchestra.run.smithi085.stdout: libsepol-devel-2.9-1.el8.x86_64 2020-11-20T13:54:50.203 INFO:teuthology.orchestra.run.smithi085.stdout: libverto-devel-0.3.0-5.el8.x86_64 2020-11-20T13:54:50.203 INFO:teuthology.orchestra.run.smithi085.stdout: libxslt-1.1.32-3.el8.x86_64 2020-11-20T13:54:50.204 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-devel-10.32-1.el8.x86_64 2020-11-20T13:54:50.204 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf16-10.32-1.el8.x86_64 2020-11-20T13:54:50.204 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf32-10.32-1.el8.x86_64 2020-11-20T13:54:50.204 INFO:teuthology.orchestra.run.smithi085.stdout: python3-chardet-3.0.4-7.el8.noarch 2020-11-20T13:54:50.204 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-11-20T13:54:50.205 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-11-20T13:54:50.205 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pysocks-1.6.8-3.el8.noarch 2020-11-20T13:54:50.205 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2020-11-20T13:54:50.205 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-11-20T13:54:50.205 INFO:teuthology.orchestra.run.smithi085.stdout: python3-urllib3-1.24.2-2.el8.noarch 2020-11-20T13:54:50.206 INFO:teuthology.orchestra.run.smithi085.stdout: zlib-devel-1.2.11-10.el8.x86_64 2020-11-20T13:54:50.206 INFO:teuthology.orchestra.run.smithi085.stdout: fmt-6.2.1-1.el8.x86_64 2020-11-20T13:54:50.206 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-11-20T13:54:50.206 INFO:teuthology.orchestra.run.smithi085.stdout: leveldb-1.22-1.el8.x86_64 2020-11-20T13:54:50.206 INFO:teuthology.orchestra.run.smithi085.stdout: liboath-2.6.2-3.el8.x86_64 2020-11-20T13:54:50.207 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind-1.3.1-3.el8.x86_64 2020-11-20T13:54:50.207 INFO:teuthology.orchestra.run.smithi085.stdout: python3-apipkg-1.5-6.el8.noarch 2020-11-20T13:54:50.207 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-11-20T13:54:50.207 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-11-20T13:54:50.207 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-11-20T13:54:50.208 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-11-20T13:54:50.208 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-11-20T13:54:50.208 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-11-20T13:54:50.208 INFO:teuthology.orchestra.run.smithi085.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-11-20T13:54:50.208 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-11-20T13:54:50.209 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-6.2-6.el8.noarch 2020-11-20T13:54:50.209 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-11-20T13:54:50.209 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-11-20T13:54:50.209 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-11-20T13:54:50.210 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-11-20T13:54:50.210 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-11-20T13:54:50.210 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend-2.6-1.el8.noarch 2020-11-20T13:54:50.210 INFO:teuthology.orchestra.run.smithi085.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-11-20T13:54:50.210 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-11-20T13:54:50.211 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes-2.4.1-12.el8.noarch 2020-11-20T13:54:50.211 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-11-20T13:54:50.211 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-11-20T13:54:50.211 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-11-20T13:54:50.211 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-11-20T13:54:50.212 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme-0.6.0-4.el8.noarch 2020-11-20T13:54:50.212 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-11-20T13:54:50.212 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-11-20T13:54:50.212 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-11-20T13:54:50.212 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-11-20T13:54:50.213 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-11-20T13:54:50.213 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-11-20T13:54:50.213 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:50.214 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2020-11-20T13:54:50.338 DEBUG:teuthology.parallel:result is None 2020-11-20T13:54:50.339 INFO:teuthology.orchestra.run.smithi117:> sudo lsb_release -is 2020-11-20T13:54:50.386 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS 2020-11-20T13:54:50.387 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:54:50.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:54:50.555 INFO:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-20T13:54:50.574 INFO:teuthology.orchestra.run.smithi117.stdout:15.2.5-588.ga7b62340.el8 2020-11-20T13:54:50.575 INFO:teuthology.packaging:The installed version of ceph is 15.2.5-588.ga7b62340.el8 2020-11-20T13:54:50.575 INFO:teuthology.task.install:The correct ceph version 15.2.5-588.ga7b62340 is installed. 2020-11-20T13:54:50.576 INFO:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2020-11-20T13:54:50.626 INFO:teuthology.orchestra.run.smithi085.stdout:CentOS 2020-11-20T13:54:50.627 DEBUG:teuthology.misc:System to be installed: CentOS 2020-11-20T13:54:50.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7b62340ad27b26642bb54ec916a97c7e1f07a34 2020-11-20T13:54:50.789 INFO:teuthology.orchestra.run.smithi085:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-11-20T13:54:50.808 INFO:teuthology.orchestra.run.smithi085.stdout:15.2.5-588.ga7b62340.el8 2020-11-20T13:54:50.809 INFO:teuthology.packaging:The installed version of ceph is 15.2.5-588.ga7b62340.el8 2020-11-20T13:54:50.809 INFO:teuthology.task.install:The correct ceph version 15.2.5-588.ga7b62340 is installed. 2020-11-20T13:54:50.810 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-20T13:54:50.810 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:54:50.810 INFO:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:54:50.824 INFO:teuthology.orchestra.run.smithi117.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:54:50.837 INFO:teuthology.orchestra.run.smithi117.stderr:24+1 records in 2020-11-20T13:54:50.837 INFO:teuthology.orchestra.run.smithi117.stderr:24+1 records out 2020-11-20T13:54:50.838 INFO:teuthology.orchestra.run.smithi117.stderr:12667 bytes (13 kB, 12 KiB) copied, 6.6758e-05 s, 190 MB/s 2020-11-20T13:54:50.839 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:54:50.839 INFO:teuthology.orchestra.run.smithi085:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:54:50.861 INFO:teuthology.orchestra.run.smithi085.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T13:54:50.876 INFO:teuthology.orchestra.run.smithi085.stderr:24+1 records in 2020-11-20T13:54:50.876 INFO:teuthology.orchestra.run.smithi085.stderr:24+1 records out 2020-11-20T13:54:50.877 INFO:teuthology.orchestra.run.smithi085.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000113303 s, 112 MB/s 2020-11-20T13:54:50.877 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-20T13:54:50.878 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:54:50.878 INFO:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:54:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:54:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:5+1 records in 2020-11-20T13:54:50.905 INFO:teuthology.orchestra.run.smithi117.stderr:5+1 records out 2020-11-20T13:54:50.905 INFO:teuthology.orchestra.run.smithi117.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 5.6842e-05 s, 53.8 MB/s 2020-11-20T13:54:50.906 INFO:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T13:54:50.973 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:54:50.974 INFO:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:54:50.987 INFO:teuthology.orchestra.run.smithi085.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T13:54:51.000 INFO:teuthology.orchestra.run.smithi085.stderr:5+1 records in 2020-11-20T13:54:51.001 INFO:teuthology.orchestra.run.smithi085.stderr:5+1 records out 2020-11-20T13:54:51.001 INFO:teuthology.orchestra.run.smithi085.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 3.5275e-05 s, 86.7 MB/s 2020-11-20T13:54:51.002 INFO:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T13:54:51.069 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-20T13:54:51.069 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:54:51.070 INFO:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:54:51.082 INFO:teuthology.orchestra.run.smithi117.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:54:51.095 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2020-11-20T13:54:51.096 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2020-11-20T13:54:51.096 INFO:teuthology.orchestra.run.smithi117.stderr:299 bytes copied, 2.605e-05 s, 11.5 MB/s 2020-11-20T13:54:51.097 INFO:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T13:54:51.163 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:54:51.164 INFO:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:54:51.176 INFO:teuthology.orchestra.run.smithi085.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T13:54:51.194 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records in 2020-11-20T13:54:51.194 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records out 2020-11-20T13:54:51.195 INFO:teuthology.orchestra.run.smithi085.stderr:299 bytes copied, 2.5949e-05 s, 11.5 MB/s 2020-11-20T13:54:51.195 INFO:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T13:54:51.261 INFO:teuthology.run_tasks:Running task ceph... 2020-11-20T13:54:51.327 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-20T13:54:51.327 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 777 /var/log/ceph 2020-11-20T13:54:51.330 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2020-11-20T13:54:51.357 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-20T13:54:51.357 INFO:teuthology.orchestra.run.smithi085:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T13:54:51.397 INFO:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T13:54:51.423 INFO:tasks.ceph:Creating extra log directories... 2020-11-20T13:54:51.424 INFO:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T13:54:51.464 INFO:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T13:54:51.490 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-11-20T13:54:51.491 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-11-20T13:54:51.492 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-11-20T13:54:51.493 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:54:51.493 INFO:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:54:51.543 INFO:teuthology.orchestra.run.smithi117.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:54:51.545 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2020-11-20T13:54:51.545 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2020-11-20T13:54:51.545 INFO:teuthology.orchestra.run.smithi117.stderr:408 bytes copied, 2.9348e-05 s, 13.9 MB/s 2020-11-20T13:54:51.545 INFO:teuthology.orchestra.run.smithi117:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T13:54:51.655 INFO:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-11-20T13:54:51.682 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:54:51.683 INFO:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:54:51.695 INFO:teuthology.orchestra.run.smithi085.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T13:54:51.696 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records in 2020-11-20T13:54:51.697 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records out 2020-11-20T13:54:51.697 INFO:teuthology.orchestra.run.smithi085.stderr:408 bytes copied, 2.6855e-05 s, 15.2 MB/s 2020-11-20T13:54:51.697 INFO:teuthology.orchestra.run.smithi085:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T13:54:51.810 INFO:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2020-11-20T13:54:51.837 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-20T13:54:51.837 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-20T13:54:51.838 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi/5628501', 'branch': 'octopus', 'description': 'fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_latest} tasks/auto-repair}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5628501', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2020-11-16_04:10:02-fs-octopus-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.1', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'cephfs_ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'octopus', 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'git://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3']], 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_octopus/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'suite_sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'a7b62340ad27b26642bb54ec916a97c7e1f07a34'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_auto_repair']}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-16_04:10:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.6029', 'targets': {'smithi117.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtDP5HjQgDih0wZKqRJqB7oDZLctFdP4o1P05xVnYWarQ1t61y1CZdNvDNjHoP+ixSrDpXbzcV0TWfkAlwYDBMYjwrMT27LrzFkjdStdDHU7E4lo9KbK4Mgu8QkkDbn92Xj46N/cWUiFNR4EVpw7G34Cu04NBFLw/aiX/mFCCsPD+ZsFnWRf45qrtHGjxtLM9p4dzIInjQ6zqpeODxq7JQKCBlU14lT9BVqeJVa1ZwqT5ShIU0cKkfC7prTyH9HNDCFnTWWqUcz52swGHB0MhavQZwwR9dnoUfOBCJXfIPzdj5A7qBFxFGGKpt/8/34T94/dmyAykmRCRdMjkAPb/ODlev89dCl0PTlQVhkMQj/DFdtaFjwdCeFY+xwpk9xKBKYfT4/sjI7YU6HS73CRa0qUM5nlBBB3mVDt7ZCn151Zb0RfDJbXSg0oEHxZHY26auIpd9DOZ5TSO5Yki3Jr5hFW93nOfn3mQKypa+h5Pv1clEF6xWv4pf8LTWHDlgcOs=', 'smithi085.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkpy7YtpzpBoIaad9dS53i37YxrKO6patHbk4vlp7vUFrs7QTpFC5a7hx+Tr+P7UTTtPXgbEurBYgJZXK3mrOh6qPzG0XbKNqCPTFgTEpIsJkZ8a2bTD9o1qijJ+bf7eust/cyhAkHQdalKUhAvzVx2OruYtRWRrsLCh+NszdDIAJ7iiyd45T25xTEnGUt5YKbJtxbqUVnSuzYe9Je0930MuB6S8NBwSw3IoDdiZX9Z4oo57B52nrDK8OdWmTLrTvkQAT+g40HSNIY+cag+gkMzYWtb3AxblMAD+NHyPa2AYTXNj8zYXNaD1EpqBrnKTkeDzNAO+728u9M1hmQbJFdjU5+/dIoPNypUxnIFXl66oj1OS0y2dUrc9LKCnS5IhLmIpOu492E4Tk71qxR5KckJres43zvHVx2ehD9kEmcN/11RniofGG5VCnCAGifuo0WpPYz3p+kw4a19Sd/0X/VikK+WK9Vz5N7ul1amAQ55HDHcfTLMZ/3mty1Fpnx6/s='}} 2020-11-20T13:54:51.839 INFO:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T13:54:51.879 INFO:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T13:54:51.893 INFO:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T13:54:51.933 INFO:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T13:54:51.961 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:54:51.962 INFO:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:54:52.014 INFO:teuthology.orchestra.run.smithi117.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:54:52.015 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_1 2020-11-20T13:54:52.015 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_2 2020-11-20T13:54:52.015 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_3 2020-11-20T13:54:52.016 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_4 2020-11-20T13:54:52.016 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2020-11-20T13:54:52.016 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2020-11-20T13:54:52.016 INFO:teuthology.orchestra.run.smithi117.stderr:72 bytes copied, 3.3078e-05 s, 2.2 MB/s 2020-11-20T13:54:52.017 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T13:54:52.017 INFO:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T13:54:52.069 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T13:54:52.069 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.069 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 59083 Links: 1 2020-11-20T13:54:52.070 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.070 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.070 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2020-11-20 13:54:47.230341775 +0000 2020-11-20T13:54:52.070 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2020-11-20 13:52:58.935719036 +0000 2020-11-20T13:54:52.070 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2020-11-20 13:52:58.935719036 +0000 2020-11-20T13:54:52.071 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2020-11-20T13:54:52.082 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2020-11-20T13:54:52.082 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2020-11-20T13:54:52.082 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000250136 s, 2.0 MB/s 2020-11-20T13:54:52.085 INFO:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T13:54:52.138 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T13:54:52.139 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.139 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 61116 Links: 1 2020-11-20T13:54:52.139 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.140 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.140 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2020-11-20 13:54:47.230341775 +0000 2020-11-20T13:54:52.140 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2020-11-20 13:52:59.271715242 +0000 2020-11-20T13:54:52.140 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2020-11-20 13:52:59.271715242 +0000 2020-11-20T13:54:52.141 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2020-11-20T13:54:52.152 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2020-11-20T13:54:52.152 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2020-11-20T13:54:52.152 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000211637 s, 2.4 MB/s 2020-11-20T13:54:52.154 INFO:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T13:54:52.208 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T13:54:52.208 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.208 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 59160 Links: 1 2020-11-20T13:54:52.209 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.209 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.209 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2020-11-20 13:54:47.230341775 +0000 2020-11-20T13:54:52.209 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2020-11-20 13:52:59.589711652 +0000 2020-11-20T13:54:52.209 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2020-11-20 13:52:59.589711652 +0000 2020-11-20T13:54:52.210 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2020-11-20T13:54:52.221 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2020-11-20T13:54:52.221 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2020-11-20T13:54:52.221 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00019849 s, 2.6 MB/s 2020-11-20T13:54:52.223 INFO:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T13:54:52.278 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T13:54:52.278 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.278 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 57188 Links: 1 2020-11-20T13:54:52.278 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.279 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.279 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2020-11-20 13:54:47.230341775 +0000 2020-11-20T13:54:52.279 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2020-11-20 13:52:59.917707949 +0000 2020-11-20T13:54:52.279 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2020-11-20 13:52:59.917707949 +0000 2020-11-20T13:54:52.280 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2020-11-20T13:54:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2020-11-20T13:54:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2020-11-20T13:54:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000160598 s, 3.2 MB/s 2020-11-20T13:54:52.293 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:54:52.293 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:54:52.294 INFO:teuthology.orchestra.run.smithi085:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:54:52.307 INFO:teuthology.orchestra.run.smithi085.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T13:54:52.308 INFO:teuthology.orchestra.run.smithi085.stdout:/dev/vg_nvme/lv_1 2020-11-20T13:54:52.309 INFO:teuthology.orchestra.run.smithi085.stdout:/dev/vg_nvme/lv_2 2020-11-20T13:54:52.309 INFO:teuthology.orchestra.run.smithi085.stdout:/dev/vg_nvme/lv_3 2020-11-20T13:54:52.309 INFO:teuthology.orchestra.run.smithi085.stdout:/dev/vg_nvme/lv_4 2020-11-20T13:54:52.309 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records in 2020-11-20T13:54:52.310 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records out 2020-11-20T13:54:52.310 INFO:teuthology.orchestra.run.smithi085.stderr:72 bytes copied, 3.3216e-05 s, 2.2 MB/s 2020-11-20T13:54:52.310 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T13:54:52.311 INFO:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T13:54:52.363 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T13:54:52.363 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.363 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 58988 Links: 1 2020-11-20T13:54:52.364 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.364 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.364 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2020-11-20 13:54:48.088040049 +0000 2020-11-20T13:54:52.364 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2020-11-20 13:52:58.943513681 +0000 2020-11-20T13:54:52.365 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2020-11-20 13:52:58.943513681 +0000 2020-11-20T13:54:52.365 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2020-11-20T13:54:52.376 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2020-11-20T13:54:52.376 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2020-11-20T13:54:52.377 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000236047 s, 2.2 MB/s 2020-11-20T13:54:52.379 INFO:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T13:54:52.433 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T13:54:52.434 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.434 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 56238 Links: 1 2020-11-20T13:54:52.434 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.434 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.435 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2020-11-20 13:54:48.088040049 +0000 2020-11-20T13:54:52.435 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2020-11-20 13:52:59.281510227 +0000 2020-11-20T13:54:52.435 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2020-11-20 13:52:59.281510227 +0000 2020-11-20T13:54:52.435 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2020-11-20T13:54:52.446 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2020-11-20T13:54:52.447 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2020-11-20T13:54:52.447 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000209691 s, 2.4 MB/s 2020-11-20T13:54:52.449 INFO:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T13:54:52.503 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T13:54:52.504 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.504 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 56303 Links: 1 2020-11-20T13:54:52.504 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.505 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.505 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2020-11-20 13:54:48.088040049 +0000 2020-11-20T13:54:52.505 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2020-11-20 13:52:59.629506671 +0000 2020-11-20T13:54:52.506 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2020-11-20 13:52:59.629506671 +0000 2020-11-20T13:54:52.506 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2020-11-20T13:54:52.516 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2020-11-20T13:54:52.516 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2020-11-20T13:54:52.517 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000189256 s, 2.7 MB/s 2020-11-20T13:54:52.518 INFO:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T13:54:52.572 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T13:54:52.573 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T13:54:52.573 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 59066 Links: 1 2020-11-20T13:54:52.574 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T13:54:52.574 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2020-11-20T13:54:52.574 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2020-11-20 13:54:48.088040049 +0000 2020-11-20T13:54:52.574 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2020-11-20 13:52:59.967503217 +0000 2020-11-20T13:54:52.575 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2020-11-20 13:52:59.967503217 +0000 2020-11-20T13:54:52.575 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2020-11-20T13:54:52.585 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2020-11-20T13:54:52.586 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2020-11-20T13:54:52.586 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000183869 s, 2.8 MB/s 2020-11-20T13:54:52.588 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:54:52.588 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T13:54:52.589 INFO:tasks.ceph:Generating config... 2020-11-20T13:54:52.591 INFO:tasks.ceph:[client] client mount timeout = 600 2020-11-20T13:54:52.592 INFO:tasks.ceph:[client] debug client = 20 2020-11-20T13:54:52.592 INFO:tasks.ceph:[client] debug ms = 1 2020-11-20T13:54:52.592 INFO:tasks.ceph:[client] pid file = 2020-11-20T13:54:52.592 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-11-20T13:54:52.593 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-11-20T13:54:52.593 INFO:tasks.ceph:[mds] debug mds = 20 2020-11-20T13:54:52.593 INFO:tasks.ceph:[mds] debug ms = 1 2020-11-20T13:54:52.594 INFO:tasks.ceph:[mds] mds bal frag = True 2020-11-20T13:54:52.594 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-11-20T13:54:52.594 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-11-20T13:54:52.595 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-11-20T13:54:52.595 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-11-20T13:54:52.595 INFO:tasks.ceph:[mds] mds debug frag = True 2020-11-20T13:54:52.595 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-11-20T13:54:52.596 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-11-20T13:54:52.596 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-11-20T13:54:52.596 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-11-20T13:54:52.597 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-11-20T13:54:52.597 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-11-20T13:54:52.597 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-20T13:54:52.597 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-20T13:54:52.598 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-20T13:54:52.598 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-20T13:54:52.598 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-20T13:54:52.599 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-11-20T13:54:52.599 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-20T13:54:52.599 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-20T13:54:52.599 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-20T13:54:52.600 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-20T13:54:52.600 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-20T13:54:52.600 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-20T13:54:52.601 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-20T13:54:52.601 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-20T13:54:52.601 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-20T13:54:52.602 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-20T13:54:52.602 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-20T13:54:52.602 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-20T13:54:52.602 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-20T13:54:52.603 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-20T13:54:52.603 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-11-20T13:54:52.603 INFO:tasks.ceph:Setting up mon.a... 2020-11-20T13:54:52.604 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-20T13:54:52.655 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.keyring 2020-11-20T13:54:52.657 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-20T13:54:52.698 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:54:52.765 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.117'), ('mon.b', '172.21.15.85'), ('mon.c', '[v2:172.21.15.85:3301,v1:172.21.15.85:6790]')] 2020-11-20T13:54:52.766 INFO:teuthology.orchestra.run.smithi117:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.117 --add b 172.21.15.85 --addv c '[v2:172.21.15.85:3301,v1:172.21.15.85:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:52.841 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:52.842 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool: generated fsid a1f0335b-4805-46c3-ba38-df4006f55611 2020-11-20T13:54:52.842 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 0 2020-11-20T13:54:52.842 INFO:teuthology.orchestra.run.smithi117.stdout:fsid a1f0335b-4805-46c3-ba38-df4006f55611 2020-11-20T13:54:52.842 INFO:teuthology.orchestra.run.smithi117.stdout:last_changed 2020-11-20T13:54:52.842143+0000 2020-11-20T13:54:52.843 INFO:teuthology.orchestra.run.smithi117.stdout:created 2020-11-20T13:54:52.842143+0000 2020-11-20T13:54:52.843 INFO:teuthology.orchestra.run.smithi117.stdout:min_mon_release 0 (unknown) 2020-11-20T13:54:52.843 INFO:teuthology.orchestra.run.smithi117.stdout:0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2020-11-20T13:54:52.843 INFO:teuthology.orchestra.run.smithi117.stdout:1: [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] mon.b 2020-11-20T13:54:52.844 INFO:teuthology.orchestra.run.smithi117.stdout:2: [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] mon.c 2020-11-20T13:54:52.844 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-20T13:54:52.845 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a1f0335b-4805-46c3-ba38-df4006f55611... 2020-11-20T13:54:52.846 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T13:54:52.849 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T13:54:52.899 INFO:teuthology.orchestra.run.smithi085.stdout:[global] 2020-11-20T13:54:52.899 INFO:teuthology.orchestra.run.smithi085.stdout: chdir = "" 2020-11-20T13:54:52.900 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T13:54:52.900 INFO:teuthology.orchestra.run.smithi085.stdout: auth supported = cephx 2020-11-20T13:54:52.900 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.900 INFO:teuthology.orchestra.run.smithi085.stdout: filestore xattr use omap = true 2020-11-20T13:54:52.900 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.901 INFO:teuthology.orchestra.run.smithi085.stdout: mon clock drift allowed = 1.000 2020-11-20T13:54:52.901 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.901 INFO:teuthology.orchestra.run.smithi085.stdout: osd crush chooseleaf type = 0 2020-11-20T13:54:52.901 INFO:teuthology.orchestra.run.smithi085.stdout: auth debug = true 2020-11-20T13:54:52.901 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.902 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on old message = true 2020-11-20T13:54:52.902 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on bug = true 2020-11-20T13:54:52.902 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.903 INFO:teuthology.orchestra.run.smithi085.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T13:54:52.903 INFO:teuthology.orchestra.run.smithi085.stdout: mon pg warn max object skew = 0 2020-11-20T13:54:52.903 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.903 INFO:teuthology.orchestra.run.smithi085.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T13:54:52.903 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T13:54:52.904 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.904 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default size = 2 2020-11-20T13:54:52.904 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.904 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow primary affinity = true 2020-11-20T13:54:52.904 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow pg remap = true 2020-11-20T13:54:52.905 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on legacy crush tunables = false 2020-11-20T13:54:52.905 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on crush straw calc version zero = false 2020-11-20T13:54:52.905 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on no sortbitwise = false 2020-11-20T13:54:52.905 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on osd down out interval zero = false 2020-11-20T13:54:52.905 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on too few osds = false 2020-11-20T13:54:52.906 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T13:54:52.906 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T13:54:52.906 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.906 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T13:54:52.906 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.907 INFO:teuthology.orchestra.run.smithi085.stdout: osd default data pool replay window = 5 2020-11-20T13:54:52.907 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.907 INFO:teuthology.orchestra.run.smithi085.stdout: mon allow pool delete = true 2020-11-20T13:54:52.907 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.907 INFO:teuthology.orchestra.run.smithi085.stdout: mon cluster log file level = debug 2020-11-20T13:54:52.907 INFO:teuthology.orchestra.run.smithi085.stdout: debug asserts on shutdown = true 2020-11-20T13:54:52.908 INFO:teuthology.orchestra.run.smithi085.stdout: mon host = "172.21.15.117,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]" 2020-11-20T13:54:52.908 INFO:teuthology.orchestra.run.smithi085.stdout: fsid = a1f0335b-4805-46c3-ba38-df4006f55611 2020-11-20T13:54:52.908 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.908 INFO:teuthology.orchestra.run.smithi085.stdout:[osd] 2020-11-20T13:54:52.908 INFO:teuthology.orchestra.run.smithi085.stdout: osd journal size = 100 2020-11-20T13:54:52.909 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.909 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub load threshold = 5.0 2020-11-20T13:54:52.909 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub max interval = 600 2020-11-20T13:54:52.909 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.909 INFO:teuthology.orchestra.run.smithi085.stdout: osd recover clone overlap = true 2020-11-20T13:54:52.910 INFO:teuthology.orchestra.run.smithi085.stdout: osd recovery max chunk = 1048576 2020-11-20T13:54:52.910 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.910 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug shutdown = true 2020-11-20T13:54:52.910 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug op order = true 2020-11-20T13:54:52.910 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify stray on activate = true 2020-11-20T13:54:52.911 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.911 INFO:teuthology.orchestra.run.smithi085.stdout: osd open classes on start = true 2020-11-20T13:54:52.911 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug pg log writeout = true 2020-11-20T13:54:52.911 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.911 INFO:teuthology.orchestra.run.smithi085.stdout: osd deep scrub update digest min age = 30 2020-11-20T13:54:52.912 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.912 INFO:teuthology.orchestra.run.smithi085.stdout: osd map max advance = 10 2020-11-20T13:54:52.912 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.912 INFO:teuthology.orchestra.run.smithi085.stdout: journal zero on create = true 2020-11-20T13:54:52.912 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.912 INFO:teuthology.orchestra.run.smithi085.stdout: filestore ondisk finisher threads = 3 2020-11-20T13:54:52.913 INFO:teuthology.orchestra.run.smithi085.stdout: filestore apply finisher threads = 3 2020-11-20T13:54:52.913 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.913 INFO:teuthology.orchestra.run.smithi085.stdout: bdev debug aio = true 2020-11-20T13:54:52.913 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug misdirected ops = true 2020-11-20T13:54:52.913 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore block size = 96636764160 2020-11-20T13:54:52.914 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore fsck on mount = True 2020-11-20T13:54:52.914 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluefs = 20 2020-11-20T13:54:52.914 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluestore = 20 2020-11-20T13:54:52.914 INFO:teuthology.orchestra.run.smithi085.stdout: debug filestore = 20 2020-11-20T13:54:52.914 INFO:teuthology.orchestra.run.smithi085.stdout: debug journal = 20 2020-11-20T13:54:52.915 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 20 2020-11-20T13:54:52.915 INFO:teuthology.orchestra.run.smithi085.stdout: debug osd = 25 2020-11-20T13:54:52.915 INFO:teuthology.orchestra.run.smithi085.stdout: debug rocksdb = 10 2020-11-20T13:54:52.915 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T13:54:52.915 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd full ratio = 0.9 2020-11-20T13:54:52.916 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd nearfull ratio = 0.8 2020-11-20T13:54:52.916 INFO:teuthology.orchestra.run.smithi085.stdout: osd failsafe full ratio = 0.95 2020-11-20T13:54:52.916 INFO:teuthology.orchestra.run.smithi085.stdout: osd objectstore = bluestore 2020-11-20T13:54:52.916 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2020-11-20T13:54:52.916 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.917 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr] 2020-11-20T13:54:52.917 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2020-11-20T13:54:52.917 INFO:teuthology.orchestra.run.smithi085.stdout: debug mgr = 20 2020-11-20T13:54:52.917 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2020-11-20T13:54:52.917 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2020-11-20T13:54:52.917 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:54:52.918 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:54:52.918 INFO:teuthology.orchestra.run.smithi085.stdout: mgr/telemetry/nag = false 2020-11-20T13:54:52.918 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.918 INFO:teuthology.orchestra.run.smithi085.stdout:[mon] 2020-11-20T13:54:52.918 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2020-11-20T13:54:52.919 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2020-11-20T13:54:52.919 INFO:teuthology.orchestra.run.smithi085.stdout: debug paxos = 20 2020-11-20T13:54:52.919 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2020-11-20T13:54:52.919 INFO:teuthology.orchestra.run.smithi085.stdout: mon data avail warn = 5 2020-11-20T13:54:52.919 INFO:teuthology.orchestra.run.smithi085.stdout: mon mgr mkfs grace = 240 2020-11-20T13:54:52.920 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:54:52.920 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd reporter subtree level = osd 2020-11-20T13:54:52.920 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd prime pg temp = true 2020-11-20T13:54:52.920 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:54:52.920 INFO:teuthology.orchestra.run.smithi085.stdout: mon op complaint time = 120 2020-11-20T13:54:52.921 INFO:teuthology.orchestra.run.smithi085.stdout: 2020-11-20T13:54:52.921 INFO:teuthology.orchestra.run.smithi085.stdout:[client] 2020-11-20T13:54:52.921 INFO:teuthology.orchestra.run.smithi085.stdout: rgw cache enabled = true 2020-11-20T13:54:52.921 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable ops log = true 2020-11-20T13:54:52.921 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable usage log = true 2020-11-20T13:54:52.922 INFO:teuthology.orchestra.run.smithi085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T13:54:52.922 INFO:teuthology.orchestra.run.smithi085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T13:54:52.922 INFO:teuthology.orchestra.run.smithi085.stdout: client mount timeout = 600 2020-11-20T13:54:52.922 INFO:teuthology.orchestra.run.smithi085.stdout: debug client = 20 2020-11-20T13:54:52.922 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2020-11-20T13:54:52.923 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = "" 2020-11-20T13:54:52.923 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2020-11-20T13:54:52.923 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2020-11-20T13:54:52.923 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.a] 2020-11-20T13:54:52.923 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.b] 2020-11-20T13:54:52.923 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.c] 2020-11-20T13:54:52.924 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2020-11-20T13:54:52.924 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2020-11-20T13:54:52.924 INFO:teuthology.orchestra.run.smithi085.stdout:[mds] 2020-11-20T13:54:52.924 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds = 20 2020-11-20T13:54:52.924 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2020-11-20T13:54:52.925 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal frag = True 2020-11-20T13:54:52.925 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal fragment size max = 10000 2020-11-20T13:54:52.925 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal merge size = 5 2020-11-20T13:54:52.925 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split bits = 3 2020-11-20T13:54:52.925 INFO:teuthology.orchestra.run.smithi085.stdout: mds bal split size = 100 2020-11-20T13:54:52.926 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug frag = True 2020-11-20T13:54:52.926 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug scatterstat = True 2020-11-20T13:54:52.926 INFO:teuthology.orchestra.run.smithi085.stdout: mds op complaint time = 180 2020-11-20T13:54:52.926 INFO:teuthology.orchestra.run.smithi085.stdout: mds verify scatter = True 2020-11-20T13:54:52.926 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2020-11-20T13:54:52.927 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 15m 2020-11-20T13:54:52.927 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 15m 2020-11-20T13:54:52.937 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2020-11-20T13:54:52.938 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2020-11-20T13:54:52.938 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T13:54:52.938 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2020-11-20T13:54:52.938 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.939 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2020-11-20T13:54:52.939 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.939 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2020-11-20T13:54:52.939 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.939 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2020-11-20T13:54:52.940 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2020-11-20T13:54:52.940 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.940 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2020-11-20T13:54:52.940 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2020-11-20T13:54:52.940 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.941 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T13:54:52.941 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2020-11-20T13:54:52.941 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.941 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T13:54:52.942 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T13:54:52.942 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.942 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2020-11-20T13:54:52.942 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.943 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2020-11-20T13:54:52.943 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2020-11-20T13:54:52.943 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2020-11-20T13:54:52.943 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2020-11-20T13:54:52.943 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2020-11-20T13:54:52.944 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2020-11-20T13:54:52.944 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2020-11-20T13:54:52.944 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T13:54:52.945 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T13:54:52.945 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.945 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T13:54:52.945 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.946 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2020-11-20T13:54:52.946 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.947 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2020-11-20T13:54:52.947 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.948 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2020-11-20T13:54:52.948 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2020-11-20T13:54:52.948 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.117,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]" 2020-11-20T13:54:52.948 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = a1f0335b-4805-46c3-ba38-df4006f55611 2020-11-20T13:54:52.948 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.949 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2020-11-20T13:54:52.949 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2020-11-20T13:54:52.949 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.949 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2020-11-20T13:54:52.949 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2020-11-20T13:54:52.950 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.950 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2020-11-20T13:54:52.950 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2020-11-20T13:54:52.950 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.950 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2020-11-20T13:54:52.951 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2020-11-20T13:54:52.951 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2020-11-20T13:54:52.951 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.951 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2020-11-20T13:54:52.951 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2020-11-20T13:54:52.952 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.952 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2020-11-20T13:54:52.952 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.952 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2020-11-20T13:54:52.952 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.953 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2020-11-20T13:54:52.953 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.953 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2020-11-20T13:54:52.953 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2020-11-20T13:54:52.954 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.954 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2020-11-20T13:54:52.954 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2020-11-20T13:54:52.954 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2020-11-20T13:54:52.954 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2020-11-20T13:54:52.955 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 20 2020-11-20T13:54:52.955 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 20 2020-11-20T13:54:52.955 INFO:teuthology.orchestra.run.smithi117.stdout: debug filestore = 20 2020-11-20T13:54:52.955 INFO:teuthology.orchestra.run.smithi117.stdout: debug journal = 20 2020-11-20T13:54:52.955 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 20 2020-11-20T13:54:52.956 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 25 2020-11-20T13:54:52.956 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 10 2020-11-20T13:54:52.956 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T13:54:52.956 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2020-11-20T13:54:52.956 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2020-11-20T13:54:52.957 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2020-11-20T13:54:52.957 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2020-11-20T13:54:52.957 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2020-11-20T13:54:52.957 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.957 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2020-11-20T13:54:52.958 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2020-11-20T13:54:52.958 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2020-11-20T13:54:52.958 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2020-11-20T13:54:52.958 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2020-11-20T13:54:52.958 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:54:52.959 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:54:52.959 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2020-11-20T13:54:52.959 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.959 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2020-11-20T13:54:52.959 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2020-11-20T13:54:52.960 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2020-11-20T13:54:52.960 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2020-11-20T13:54:52.960 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2020-11-20T13:54:52.960 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2020-11-20T13:54:52.960 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2020-11-20T13:54:52.961 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2020-11-20T13:54:52.961 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2020-11-20T13:54:52.961 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2020-11-20T13:54:52.961 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2020-11-20T13:54:52.961 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2020-11-20T13:54:52.962 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:54:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2020-11-20T13:54:52.962 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2020-11-20T13:54:52.962 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2020-11-20T13:54:52.962 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2020-11-20T13:54:52.963 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T13:54:52.963 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T13:54:52.963 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2020-11-20T13:54:52.963 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2020-11-20T13:54:52.963 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2020-11-20T13:54:52.964 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2020-11-20T13:54:52.964 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 15m 2020-11-20T13:54:52.964 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 15m 2020-11-20T13:54:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2020-11-20T13:54:52.964 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2020-11-20T13:54:52.965 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2020-11-20T13:54:52.965 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2020-11-20T13:54:52.965 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2020-11-20T13:54:52.965 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2020-11-20T13:54:52.965 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2020-11-20T13:54:52.966 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2020-11-20T13:54:52.966 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal frag = True 2020-11-20T13:54:52.966 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal fragment size max = 10000 2020-11-20T13:54:52.966 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal merge size = 5 2020-11-20T13:54:52.966 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal split bits = 3 2020-11-20T13:54:52.967 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal split size = 100 2020-11-20T13:54:52.967 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2020-11-20T13:54:52.967 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = True 2020-11-20T13:54:52.967 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2020-11-20T13:54:52.967 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = True 2020-11-20T13:54:52.968 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2020-11-20T13:54:52.968 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 15m 2020-11-20T13:54:52.968 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 15m 2020-11-20T13:54:52.969 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-20T13:54:52.970 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-20T13:54:53.019 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-20T13:54:53.036 DEBUG:teuthology.orchestra.remote:smithi117:/etc/ceph/ceph.keyring is 206B 2020-11-20T13:54:53.129 DEBUG:teuthology.orchestra.remote:smithi117:/home/ubuntu/cephtest/ceph.monmap is 379B 2020-11-20T13:54:53.177 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2020-11-20T13:54:53.178 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:54:53.178 INFO:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:54:53.179 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:54:53.191 INFO:teuthology.orchestra.run.smithi117.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:54:53.204 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2020-11-20T13:54:53.204 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2020-11-20T13:54:53.205 INFO:teuthology.orchestra.run.smithi117.stderr:206 bytes copied, 4.8962e-05 s, 4.2 MB/s 2020-11-20T13:54:53.205 INFO:teuthology.orchestra.run.smithi117.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:54:53.218 INFO:teuthology.orchestra.run.smithi117:> set -ex 2020-11-20T13:54:53.219 INFO:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:53.273 INFO:teuthology.orchestra.run.smithi117.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:53.273 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2020-11-20T13:54:53.273 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2020-11-20T13:54:53.273 INFO:teuthology.orchestra.run.smithi117.stderr:379 bytes copied, 4.435e-05 s, 8.5 MB/s 2020-11-20T13:54:53.274 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi085.front.sepia.ceph.com 2020-11-20T13:54:53.274 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:54:53.274 INFO:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:54:53.275 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:54:53.288 INFO:teuthology.orchestra.run.smithi085.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T13:54:53.301 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records in 2020-11-20T13:54:53.302 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records out 2020-11-20T13:54:53.302 INFO:teuthology.orchestra.run.smithi085.stderr:206 bytes copied, 2.9994e-05 s, 6.9 MB/s 2020-11-20T13:54:53.302 INFO:teuthology.orchestra.run.smithi085.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T13:54:53.317 INFO:teuthology.orchestra.run.smithi085:> set -ex 2020-11-20T13:54:53.317 INFO:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:53.369 INFO:teuthology.orchestra.run.smithi085.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:53.370 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records in 2020-11-20T13:54:53.370 INFO:teuthology.orchestra.run.smithi085.stderr:0+1 records out 2020-11-20T13:54:53.371 INFO:teuthology.orchestra.run.smithi085.stderr:379 bytes copied, 2.9472e-05 s, 12.9 MB/s 2020-11-20T13:54:53.371 INFO:tasks.ceph:Setting up mon nodes... 2020-11-20T13:54:53.371 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-20T13:54:53.372 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T13:54:53.424 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T13:54:53.426 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T13:54:53.494 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T13:54:53.496 INFO:tasks.ceph:Setting up mds nodes... 2020-11-20T13:54:53.497 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T13:54:53.550 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T13:54:53.552 INFO:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-20T13:54:53.578 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T13:54:53.631 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T13:54:53.633 INFO:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-11-20T13:54:53.660 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-20T13:54:53.661 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-20T13:54:53.701 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-20T13:54:53.717 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-11-20T13:54:53.796 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-11-20T13:54:53.811 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2020-11-20T13:54:53.851 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.2.keyring 2020-11-20T13:54:53.867 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2020-11-20T13:54:53.946 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.3.keyring 2020-11-20T13:54:53.961 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-20T13:54:53.962 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T13:54:53.962 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-20T13:54:53.989 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:54:53.990 INFO:tasks.ceph:role: osd.0 2020-11-20T13:54:53.990 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2020-11-20T13:54:53.990 INFO:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T13:54:54.523 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:54:54.524 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:54:54.524 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:54:54.524 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2020-11-20T13:54:54.524 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:54:54.524 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:54:54.525 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:54:54.525 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:54:54.525 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:54:54.525 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:54:54.526 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2020-11-20T13:54:54.527 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-20T13:54:54.656 INFO:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-20T13:54:54.687 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-20T13:54:54.753 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:54:54.754 INFO:tasks.ceph:role: osd.1 2020-11-20T13:54:54.754 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2020-11-20T13:54:54.754 INFO:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T13:54:55.265 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:54:55.266 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:54:55.266 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:54:55.266 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2020-11-20T13:54:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:54:55.267 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:54:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:54:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:54:55.267 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:54:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:54:55.268 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2020-11-20T13:54:55.269 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-20T13:54:55.315 INFO:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-20T13:54:55.346 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-20T13:54:55.412 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:54:55.412 INFO:tasks.ceph:role: osd.2 2020-11-20T13:54:55.413 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2020-11-20T13:54:55.413 INFO:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T13:54:55.944 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:54:55.945 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:54:55.945 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:54:55.945 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2020-11-20T13:54:55.945 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:54:55.946 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:54:55.946 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:54:55.946 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:54:55.946 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:54:55.947 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:54:55.947 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2020-11-20T13:54:55.948 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-20T13:54:56.013 INFO:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-20T13:54:56.044 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-20T13:54:56.110 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T13:54:56.110 INFO:tasks.ceph:role: osd.3 2020-11-20T13:54:56.110 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com 2020-11-20T13:54:56.111 INFO:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T13:54:56.638 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:54:56.639 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:54:56.639 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:54:56.639 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2020-11-20T13:54:56.639 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:54:56.640 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:54:56.640 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:54:56.640 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:54:56.640 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:54:56.641 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:54:56.641 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2020-11-20T13:54:56.642 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-20T13:54:56.733 INFO:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-20T13:54:56.763 INFO:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:56.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:56.846+0000 7fcf0c093ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-20T13:54:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:56.848+0000 7fcf0c093ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-20T13:54:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:56.848+0000 7fcf0c093ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T13:54:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:56.848+0000 7fcf0c093ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T13:54:56.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:56.848+0000 7fcf0c093ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T13:54:56.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:56.855+0000 7fcf0c093ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-20T13:54:56.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:56.884+0000 7fcf0c093ec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:54:59.190 INFO:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-20T13:54:59.260 INFO:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:54:59.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:59.346+0000 7f29b70beec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-20T13:54:59.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:59.347+0000 7f29b70beec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-20T13:54:59.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:59.347+0000 7f29b70beec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T13:54:59.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:59.347+0000 7f29b70beec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T13:54:59.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:59.347+0000 7f29b70beec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T13:54:59.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:59.347+0000 7f29b70beec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-20T13:54:59.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:54:59.357+0000 7f29b70beec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:01.690 INFO:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-20T13:55:01.722 INFO:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:01.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:01.810+0000 7fb5e1f3bec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-20T13:55:01.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:01.810+0000 7fb5e1f3bec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-20T13:55:01.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:01.810+0000 7fb5e1f3bec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T13:55:01.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:01.810+0000 7fb5e1f3bec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T13:55:01.812 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:01.810+0000 7fb5e1f3bec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T13:55:01.812 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:01.811+0000 7fb5e1f3bec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-20T13:55:01.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:01.825+0000 7fb5e1f3bec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:04.150 INFO:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-20T13:55:04.181 INFO:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:04.271 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:04.270+0000 7ff28b0efec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-20T13:55:04.271 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:04.271+0000 7ff28b0efec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-20T13:55:04.271 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:04.271+0000 7ff28b0efec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T13:55:04.272 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:04.271+0000 7ff28b0efec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T13:55:04.272 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:04.271+0000 7ff28b0efec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T13:55:04.273 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:04.271+0000 7ff28b0efec0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-20T13:55:04.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:04.285+0000 7ff28b0efec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:06.611 INFO:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-20T13:55:06.642 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:06.676 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:06.676 INFO:tasks.ceph:role: osd.4 2020-11-20T13:55:06.677 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com 2020-11-20T13:55:06.677 INFO:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T13:55:07.203 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:07.203 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:07.203 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:07.203 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 2020-11-20T13:55:07.204 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:07.204 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:07.204 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:07.204 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:07.204 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:07.205 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:07.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2020-11-20T13:55:07.206 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:07.321 INFO:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:07.354 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:07.422 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:07.422 INFO:tasks.ceph:role: osd.5 2020-11-20T13:55:07.422 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com 2020-11-20T13:55:07.423 INFO:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T13:55:07.954 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:07.954 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:07.954 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:07.955 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 2020-11-20T13:55:07.955 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:07.955 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:07.955 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:07.955 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:07.956 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:07.956 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:07.957 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2020-11-20T13:55:07.957 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:08.007 INFO:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:08.039 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:08.108 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:08.108 INFO:tasks.ceph:role: osd.6 2020-11-20T13:55:08.108 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com 2020-11-20T13:55:08.109 INFO:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T13:55:08.647 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:08.648 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:08.648 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:08.648 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 2020-11-20T13:55:08.648 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:08.649 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:08.649 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:08.649 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:08.649 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:08.650 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:08.650 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2020-11-20T13:55:08.651 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:08.722 INFO:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:08.755 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:08.823 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T13:55:08.824 INFO:tasks.ceph:role: osd.7 2020-11-20T13:55:08.824 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com 2020-11-20T13:55:08.824 INFO:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T13:55:09.366 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T13:55:09.366 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T13:55:09.366 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-11-20T13:55:09.367 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 2020-11-20T13:55:09.367 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T13:55:09.367 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2020-11-20T13:55:09.367 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-11-20T13:55:09.367 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T13:55:09.368 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T13:55:09.368 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T13:55:09.369 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2020-11-20T13:55:09.369 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:09.468 INFO:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:09.499 INFO:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:09.582 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:09.581+0000 7f5dcef56ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-20T13:55:09.585 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:09.585+0000 7f5dcef56ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-20T13:55:09.586 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:09.585+0000 7f5dcef56ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T13:55:09.586 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:09.585+0000 7f5dcef56ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T13:55:09.586 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:09.585+0000 7f5dcef56ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T13:55:09.590 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:09.589+0000 7f5dcef56ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-20T13:55:09.617 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:09.617+0000 7f5dcef56ec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:11.930 INFO:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-20T13:55:12.002 INFO:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:12.092 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:12.091+0000 7f214815fec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-20T13:55:12.092 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:12.091+0000 7f214815fec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-20T13:55:12.093 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:12.091+0000 7f214815fec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T13:55:12.093 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:12.091+0000 7f214815fec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T13:55:12.093 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:12.091+0000 7f214815fec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T13:55:12.093 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:12.092+0000 7f214815fec0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-20T13:55:12.106 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:12.106+0000 7f214815fec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:14.436 INFO:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-20T13:55:14.468 INFO:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:14.558 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:14.556+0000 7f7544fcaec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-20T13:55:14.558 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:14.556+0000 7f7544fcaec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-20T13:55:14.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:14.556+0000 7f7544fcaec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T13:55:14.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:14.556+0000 7f7544fcaec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T13:55:14.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:14.556+0000 7f7544fcaec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T13:55:14.560 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:14.557+0000 7f7544fcaec0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-20T13:55:14.573 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:14.570+0000 7f7544fcaec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:16.907 INFO:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-20T13:55:16.939 INFO:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:17.027 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:17.026+0000 7fdf2601aec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-20T13:55:17.028 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:17.026+0000 7fdf2601aec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-20T13:55:17.029 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:17.026+0000 7fdf2601aec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T13:55:17.029 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:17.026+0000 7fdf2601aec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T13:55:17.029 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:17.026+0000 7fdf2601aec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T13:55:17.030 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:17.027+0000 7fdf2601aec0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-20T13:55:17.041 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:17.040+0000 7fdf2601aec0 -1 freelist read_size_meta_from_db missing size meta in DB 2020-11-20T13:55:19.367 INFO:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-20T13:55:19.400 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-20T13:55:19.401 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:19.419 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.TXxo5QTVuw 2020-11-20T13:55:19.419 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.TXxo5QTVuw 2020-11-20T13:55:19.491 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.TXxo5QTVuw 2020-11-20T13:55:19.622 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.TXxo5QTVuw is 56B 2020-11-20T13:55:19.671 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.TXxo5QTVuw 2020-11-20T13:55:19.686 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:19.740 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.e4eIXXk8ZF 2020-11-20T13:55:19.741 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.e4eIXXk8ZF 2020-11-20T13:55:19.809 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.e4eIXXk8ZF 2020-11-20T13:55:19.935 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.e4eIXXk8ZF is 56B 2020-11-20T13:55:19.983 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.e4eIXXk8ZF 2020-11-20T13:55:19.997 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:20.050 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.Z9Ztv4vZcB 2020-11-20T13:55:20.050 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.Z9Ztv4vZcB 2020-11-20T13:55:20.118 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.Z9Ztv4vZcB 2020-11-20T13:55:20.242 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.Z9Ztv4vZcB is 56B 2020-11-20T13:55:20.290 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.Z9Ztv4vZcB 2020-11-20T13:55:20.304 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:20.357 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.IRxxsYhqBd 2020-11-20T13:55:20.358 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.IRxxsYhqBd 2020-11-20T13:55:20.426 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.IRxxsYhqBd 2020-11-20T13:55:20.549 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.IRxxsYhqBd is 56B 2020-11-20T13:55:20.596 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.IRxxsYhqBd 2020-11-20T13:55:20.610 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:20.663 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.ix1DlAW70Z 2020-11-20T13:55:20.664 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.ix1DlAW70Z 2020-11-20T13:55:20.730 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.ix1DlAW70Z 2020-11-20T13:55:20.853 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.ix1DlAW70Z is 56B 2020-11-20T13:55:20.901 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.ix1DlAW70Z 2020-11-20T13:55:20.914 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:20.967 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.vNoc2WaaOv 2020-11-20T13:55:20.967 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.vNoc2WaaOv 2020-11-20T13:55:21.033 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.vNoc2WaaOv 2020-11-20T13:55:21.156 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.vNoc2WaaOv is 56B 2020-11-20T13:55:21.206 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.vNoc2WaaOv 2020-11-20T13:55:21.220 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:21.238 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.fChbAOYSrb 2020-11-20T13:55:21.239 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.fChbAOYSrb 2020-11-20T13:55:21.310 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.fChbAOYSrb 2020-11-20T13:55:21.441 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.fChbAOYSrb is 56B 2020-11-20T13:55:21.489 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.fChbAOYSrb 2020-11-20T13:55:21.504 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:21.559 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.2G0TAIoqjK 2020-11-20T13:55:21.560 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.2G0TAIoqjK 2020-11-20T13:55:21.629 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.2G0TAIoqjK 2020-11-20T13:55:21.755 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.2G0TAIoqjK is 56B 2020-11-20T13:55:21.805 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.2G0TAIoqjK 2020-11-20T13:55:21.819 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:21.840 INFO:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:21.862 INFO:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:21.873 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.rwRAGHLGWs 2020-11-20T13:55:21.874 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.rwRAGHLGWs 2020-11-20T13:55:21.941 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.rwRAGHLGWs 2020-11-20T13:55:22.070 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.rwRAGHLGWs is 56B 2020-11-20T13:55:22.123 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.rwRAGHLGWs 2020-11-20T13:55:22.137 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:22.190 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.r8692XjjBj 2020-11-20T13:55:22.190 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.r8692XjjBj 2020-11-20T13:55:22.258 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.r8692XjjBj 2020-11-20T13:55:22.381 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.r8692XjjBj is 56B 2020-11-20T13:55:22.429 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.r8692XjjBj 2020-11-20T13:55:22.442 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:22.495 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.rFLGnes18g 2020-11-20T13:55:22.496 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.rFLGnes18g 2020-11-20T13:55:22.562 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.rFLGnes18g 2020-11-20T13:55:22.687 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.rFLGnes18g is 56B 2020-11-20T13:55:22.735 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.rFLGnes18g 2020-11-20T13:55:22.749 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:22.802 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.CzZ1rILoKr 2020-11-20T13:55:22.802 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.CzZ1rILoKr 2020-11-20T13:55:22.869 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.CzZ1rILoKr 2020-11-20T13:55:22.992 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.CzZ1rILoKr is 56B 2020-11-20T13:55:23.041 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.CzZ1rILoKr 2020-11-20T13:55:23.073 DEBUG:teuthology.orchestra.remote:smithi117:/etc/ceph/ceph.client.0.keyring is 59B 2020-11-20T13:55:23.167 DEBUG:teuthology.orchestra.remote:smithi117:/etc/ceph/ceph.client.1.keyring is 59B 2020-11-20T13:55:23.232 DEBUG:teuthology.orchestra.remote:smithi085:/etc/ceph/ceph.client.2.keyring is 59B 2020-11-20T13:55:23.325 DEBUG:teuthology.orchestra.remote:smithi085:/etc/ceph/ceph.client.3.keyring is 59B 2020-11-20T13:55:23.373 INFO:tasks.ceph:Adding keys to all mons... 2020-11-20T13:55:23.374 INFO:teuthology.orchestra.run.smithi085:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T13:55:23.376 INFO:teuthology.orchestra.run.smithi117:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T13:55:23.418 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.y] 2020-11-20T13:55:23.418 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfo6hOGRAAcImFrplXnFTTA0YGGvmJ8g== 2020-11-20T13:55:23.418 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2020-11-20T13:55:23.419 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfEKfQIBAABP6nMcAOErqsnKUF6ZSfsg== 2020-11-20T13:55:23.419 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.0] 2020-11-20T13:55:23.419 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCwyrdfY2R8MhAAVs+5CcEQIMFK/paz/FYP/A== 2020-11-20T13:55:23.419 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.1] 2020-11-20T13:55:23.419 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCzyrdf82K1FBAAlG7E19KJBgdvlws3jqAX7Q== 2020-11-20T13:55:23.420 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.2] 2020-11-20T13:55:23.420 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC1yrdf2+NXMBAAsjWqWp3/hIRz2ZW5N6GBQg== 2020-11-20T13:55:23.420 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.3] 2020-11-20T13:55:23.420 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC4yrdf+pcsEBAAcq3g4vKVejtTK4xPJCnHQw== 2020-11-20T13:55:23.420 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.x] 2020-11-20T13:55:23.421 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfqaJ8HRAAvESNIwCSNYWQq2xSySnCtQ== 2020-11-20T13:55:23.421 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2020-11-20T13:55:23.421 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfwdmmJRAAKx2MfVYCS9Q3q4qz47XB7Q== 2020-11-20T13:55:23.421 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.4] 2020-11-20T13:55:23.422 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQC9yrdfbLG4IhAA8BapeiLCGll6S8dzis39OQ== 2020-11-20T13:55:23.422 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.5] 2020-11-20T13:55:23.422 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDAyrdfixR6BRAAaztS0ZrTIwuMvnazHISWuQ== 2020-11-20T13:55:23.422 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.6] 2020-11-20T13:55:23.423 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDCyrdffY00IRAA7l+trbmTc2xXBAOQ3mjFcA== 2020-11-20T13:55:23.423 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.7] 2020-11-20T13:55:23.423 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQDFyrdf17WiARAAUfSZHp54qwZ98EB559uhqw== 2020-11-20T13:55:23.423 INFO:teuthology.orchestra.run.smithi085.stdout:[client.0] 2020-11-20T13:55:23.423 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfQ6vdKRAAAom8Z8dJNDxlDSnrjb9q2g== 2020-11-20T13:55:23.424 INFO:teuthology.orchestra.run.smithi085.stdout:[client.1] 2020-11-20T13:55:23.424 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfp8qALxAAIklppc7VXYVKg8gwnCc5ww== 2020-11-20T13:55:23.424 INFO:teuthology.orchestra.run.smithi085.stdout:[client.2] 2020-11-20T13:55:23.424 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfmQrDMhAAm5c++eZqE35+xdgSAY8SPg== 2020-11-20T13:55:23.424 INFO:teuthology.orchestra.run.smithi085.stdout:[client.3] 2020-11-20T13:55:23.425 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQCtyrdfUINnOBAACZQKMu1Q/kkZKb7j0qhM9A== 2020-11-20T13:55:23.426 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.y] 2020-11-20T13:55:23.426 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfo6hOGRAAcImFrplXnFTTA0YGGvmJ8g== 2020-11-20T13:55:23.426 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2020-11-20T13:55:23.427 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfEKfQIBAABP6nMcAOErqsnKUF6ZSfsg== 2020-11-20T13:55:23.427 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2020-11-20T13:55:23.427 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCwyrdfY2R8MhAAVs+5CcEQIMFK/paz/FYP/A== 2020-11-20T13:55:23.427 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2020-11-20T13:55:23.427 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCzyrdf82K1FBAAlG7E19KJBgdvlws3jqAX7Q== 2020-11-20T13:55:23.428 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2020-11-20T13:55:23.428 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQC1yrdf2+NXMBAAsjWqWp3/hIRz2ZW5N6GBQg== 2020-11-20T13:55:23.428 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.3] 2020-11-20T13:55:23.428 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQC4yrdf+pcsEBAAcq3g4vKVejtTK4xPJCnHQw== 2020-11-20T13:55:23.429 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.x] 2020-11-20T13:55:23.429 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfqaJ8HRAAvESNIwCSNYWQq2xSySnCtQ== 2020-11-20T13:55:23.429 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2020-11-20T13:55:23.429 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfwdmmJRAAKx2MfVYCS9Q3q4qz47XB7Q== 2020-11-20T13:55:23.429 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.4] 2020-11-20T13:55:23.430 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQC9yrdfbLG4IhAA8BapeiLCGll6S8dzis39OQ== 2020-11-20T13:55:23.430 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.5] 2020-11-20T13:55:23.430 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDAyrdfixR6BRAAaztS0ZrTIwuMvnazHISWuQ== 2020-11-20T13:55:23.430 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.6] 2020-11-20T13:55:23.430 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDCyrdffY00IRAA7l+trbmTc2xXBAOQ3mjFcA== 2020-11-20T13:55:23.431 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.7] 2020-11-20T13:55:23.431 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDFyrdf17WiARAAUfSZHp54qwZ98EB559uhqw== 2020-11-20T13:55:23.431 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2020-11-20T13:55:23.431 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfQ6vdKRAAAom8Z8dJNDxlDSnrjb9q2g== 2020-11-20T13:55:23.432 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2020-11-20T13:55:23.432 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfp8qALxAAIklppc7VXYVKg8gwnCc5ww== 2020-11-20T13:55:23.432 INFO:teuthology.orchestra.run.smithi117.stdout:[client.2] 2020-11-20T13:55:23.432 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfmQrDMhAAm5c++eZqE35+xdgSAY8SPg== 2020-11-20T13:55:23.432 INFO:teuthology.orchestra.run.smithi117.stdout:[client.3] 2020-11-20T13:55:23.433 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCtyrdfUINnOBAACZQKMu1Q/kkZKb7j0qhM9A== 2020-11-20T13:55:23.433 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:55:23.468 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:55:23.522 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:55:23.524 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:55:23.576 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.579 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.627 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.631 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.679 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.682 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.729 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.732 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.779 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:55:23.782 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T13:55:23.829 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:55:23.836 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T13:55:23.886 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.888 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.936 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.938 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.985 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:23.987 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:24.036 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:24.038 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T13:55:24.085 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.088 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.138 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.141 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.192 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.195 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.245 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.247 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T13:55:24.297 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-20T13:55:24.298 INFO:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-20T13:55:24.326 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T13:55:24.419 INFO:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-20T13:55:24.446 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-20T13:55:24.474 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T13:55:24.567 INFO:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-20T13:55:24.594 INFO:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-20T13:55:24.660 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T13:55:24.753 INFO:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-20T13:55:24.780 INFO:teuthology.orchestra.run.smithi085:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:24.823 INFO:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T13:55:24.838 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-20T13:55:24.838 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T13:55:24.839 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T13:55:24.880 INFO:tasks.ceph.mon.a:Started 2020-11-20T13:55:24.881 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T13:55:24.881 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T13:55:24.883 INFO:tasks.ceph.mon.b:Started 2020-11-20T13:55:24.884 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-20T13:55:24.884 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-20T13:55:24.886 INFO:tasks.ceph.mon.c:Started 2020-11-20T13:55:24.886 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-20T13:55:24.887 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-20T13:55:24.887 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-20T13:55:24.889 INFO:tasks.ceph.mgr.y:Started 2020-11-20T13:55:24.889 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-20T13:55:24.889 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-20T13:55:24.891 INFO:tasks.ceph.mgr.x:Started 2020-11-20T13:55:24.892 INFO:tasks.ceph:Setting crush tunables to default 2020-11-20T13:55:24.892 INFO:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd crush tunables default 2020-11-20T13:55:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.001+0000 7fbb487e1700 1 Processor -- start 2020-11-20T13:55:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.001+0000 7fbb487e1700 1 -- start start 2020-11-20T13:55:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb487e1700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.227 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb487e1700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.228 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb487e1700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb400995e0 con 0x7fbb4005f300 2020-11-20T13:55:30.228 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb487e1700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400993c0 con 0x7fbb40099fc0 2020-11-20T13:55:30.228 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb487e1700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40099880 con 0x7fbb400b59a0 2020-11-20T13:55:30.228 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb4657d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb4757f700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb4657d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:39496/0 (socket says 172.21.15.117:39496) 2020-11-20T13:55:30.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb4757f700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:51056/0 (socket says 172.21.15.117:51056) 2020-11-20T13:55:30.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb46d7e700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7fbb4005f300 0x7fbb4009b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:42494/0 (socket says 172.21.15.117:42494) 2020-11-20T13:55:30.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.002+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 learned_addr learned my addr 172.21.15.117:0/1703486041 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:30.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T13:55:30.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T13:55:30.232 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.003+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T13:55:30.232 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.232 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.232 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.232 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T13:55:30.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T13:55:30.234 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.204+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T13:55:30.234 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.234 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.234 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T13:55:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T13:55:30.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:25.605+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T13:55:30.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T13:55:30.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T13:55:30.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:26.406+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T13:55:30.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb40098ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T13:55:30.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb40098ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 msgr2=0x7fbb4009a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T13:55:30.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.85:3300/0 conn(0x7fbb40099fc0 0x7fbb4009a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 -- 172.21.15.117:0/1703486041 >> v1:172.21.15.85:6789/0 conn(0x7fbb4005f300 legacy=0x7fbb4009b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:30.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 -- 172.21.15.117:0/1703486041 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb40099b20 con 0x7fbb240033a0 2020-11-20T13:55:30.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb44d7a700 1 -- 172.21.15.117:0/1703486041 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400993c0 con 0x7fbb24007050 2020-11-20T13:55:30.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.003+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T13:55:30.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.204+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.204+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.204+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.204+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.204+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T13:55:30.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.605+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.243 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.605+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.243 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.605+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.243 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.605+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.243 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:28.605+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T13:55:30.243 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:29.407+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.244 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:29.407+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T13:55:30.244 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:29.407+0000 7fbb4657d700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T13:55:30.244 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:29.407+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T13:55:30.244 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:29.407+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T13:55:30.244 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.214+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973114059 0 0) 0x7fbb40099b20 con 0x7fbb240033a0 2020-11-20T13:55:30.245 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.214+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2c003230 con 0x7fbb240033a0 2020-11-20T13:55:30.245 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4232951490 0 0) 0x7fbb2c003230 con 0x7fbb240033a0 2020-11-20T13:55:30.245 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 msgr2=0x7fbb240094f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T13:55:30.245 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 --2- 172.21.15.117:0/1703486041 >> v2:172.21.15.117:3300/0 conn(0x7fbb24007050 0x7fbb240094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.245 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2400ac00 con 0x7fbb240033a0 2020-11-20T13:55:30.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fbb3c0016b0 con 0x7fbb240033a0 2020-11-20T13:55:30.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 >> v1:172.21.15.117:6789/0 conn(0x7fbb240033a0 legacy=0x7fbb24005820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:30.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 reap_dead start 2020-11-20T13:55:30.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb2c003770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb2c003cb0 0x7fbb2c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb2c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400993c0 con 0x7fbb2c003cb0 2020-11-20T13:55:30.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c004240 con 0x7fbb2c0049c0 2020-11-20T13:55:30.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2c00b720 con 0x7fbb400b59a0 2020-11-20T13:55:30.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb2c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb46d7e700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb2c003770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4657d700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb2c003cb0 0x7fbb2c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 msgr2=0x7fbb2c003770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:30.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb2c003770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.249 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb2c003cb0 msgr2=0x7fbb2c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:30.249 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb2c003cb0 0x7fbb2c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.249 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 -- 172.21.15.117:0/1703486041 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c003230 con 0x7fbb2c0049c0 2020-11-20T13:55:30.249 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb46d7e700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb2c003770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:30.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.220+0000 7fbb4757f700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb2c004e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fbb3800a000 tx=0x7fbb38008040).ready entity=mon.1 client_cookie=e18045d5d2d429a8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:30.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.221+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb38058d00 con 0x7fbb2c0049c0 2020-11-20T13:55:30.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.221+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3800c040 con 0x7fbb2c0049c0 2020-11-20T13:55:30.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb45d7c700 1 -- 172.21.15.117:0/1703486041 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb38012600 con 0x7fbb2c0049c0 2020-11-20T13:55:30.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 -- 172.21.15.117:0/1703486041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 msgr2=0x7fbb2c004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:30.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb2c004e40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fbb3800a000 tx=0x7fbb38008040).stop 2020-11-20T13:55:30.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 -- 172.21.15.117:0/1703486041 shutdown_connections 2020-11-20T13:55:30.252 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb2c004e40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.252 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb2c003cb0 0x7fbb2c0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.253 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 --2- 172.21.15.117:0/1703486041 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb400b59a0 0x7fbb2c003770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.253 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 -- 172.21.15.117:0/1703486041 >> 172.21.15.117:0/1703486041 conn(0x7fbb400b9490 msgr2=0x7fbb400b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:30.253 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 -- 172.21.15.117:0/1703486041 shutdown_connections 2020-11-20T13:55:30.253 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.222+0000 7fbb487e1700 1 -- 172.21.15.117:0/1703486041 wait complete. 2020-11-20T13:55:30.254 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.224+0000 7fbb487e1700 1 Processor -- start 2020-11-20T13:55:30.256 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.224+0000 7fbb487e1700 1 -- start start 2020-11-20T13:55:30.256 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb487e1700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb2c003cb0 0x7fbb40148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.257 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb487e1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb400af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.257 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb487e1700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb400b59a0 0x7fbb400925b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:30.257 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb487e1700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400b55e0 con 0x7fbb400b59a0 2020-11-20T13:55:30.257 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb487e1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400b5340 con 0x7fbb2c0049c0 2020-11-20T13:55:30.258 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb487e1700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40092110 con 0x7fbb2c003cb0 2020-11-20T13:55:30.258 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb400af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.258 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb400af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39494/0 (socket says 172.21.15.117:39494) 2020-11-20T13:55:30.258 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb46d7e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb2c003cb0 0x7fbb40148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.258 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 -- 172.21.15.117:0/2906266799 learned_addr learned my addr 172.21.15.117:0/2906266799 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:30.259 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4757f700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb400b59a0 0x7fbb400925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:30.259 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 -- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb2c003cb0 msgr2=0x7fbb40148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:30.259 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb2c003cb0 0x7fbb40148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.259 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 -- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb400b59a0 msgr2=0x7fbb400925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:30.259 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb400b59a0 0x7fbb400925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.260 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4657d700 1 -- 172.21.15.117:0/2906266799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400b03c0 con 0x7fbb2c0049c0 2020-11-20T13:55:30.260 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.225+0000 7fbb4757f700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb400b59a0 0x7fbb400925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:30.260 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb4657d700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb400af880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbb3000a000 tx=0x7fbb30008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:30.260 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb377fe700 1 -- 172.21.15.117:0/2906266799 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb30016d40 con 0x7fbb2c0049c0 2020-11-20T13:55:30.260 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb377fe700 1 -- 172.21.15.117:0/2906266799 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3001f040 con 0x7fbb2c0049c0 2020-11-20T13:55:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb400b4f80 con 0x7fbb2c0049c0 2020-11-20T13:55:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb377fe700 1 -- 172.21.15.117:0/2906266799 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb300245f0 con 0x7fbb2c0049c0 2020-11-20T13:55:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb40092ce0 con 0x7fbb2c0049c0 2020-11-20T13:55:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb377fe700 1 -- 172.21.15.117:0/2906266799 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fbb30012040 con 0x7fbb2c0049c0 2020-11-20T13:55:30.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.226+0000 7fbb377fe700 1 -- 172.21.15.117:0/2906266799 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fbb3002e7b0 con 0x7fbb2c0049c0 2020-11-20T13:55:30.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.227+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb400b5190 con 0x7fbb2c0049c0 2020-11-20T13:55:30.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.229+0000 7fbb377fe700 1 -- 172.21.15.117:0/2906266799 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fbb30016ea0 con 0x7fbb2c0049c0 2020-11-20T13:55:30.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.306+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbb400922d0 con 0x7fbb2c0049c0 2020-11-20T13:55:30.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb377fe700 1 -- 172.21.15.117:0/2906266799 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbb30024dd0 con 0x7fbb2c0049c0 2020-11-20T13:55:30.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 msgr2=0x7fbb400af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:30.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb487e1700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb400af880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbb3000a000 tx=0x7fbb30008040).stop 2020-11-20T13:55:30.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 shutdown_connections 2020-11-20T13:55:30.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb487e1700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fbb2c003cb0 0x7fbb40148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb487e1700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fbb400b59a0 0x7fbb400925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb487e1700 1 --2- 172.21.15.117:0/2906266799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2c0049c0 0x7fbb400af880 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:30.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.314+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 >> 172.21.15.117:0/2906266799 conn(0x7fbb400b9490 msgr2=0x7fbb400b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:30.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.315+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 shutdown_connections 2020-11-20T13:55:30.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:30.315+0000 7fbb487e1700 1 -- 172.21.15.117:0/2906266799 wait complete. 2020-11-20T13:55:30.318 INFO:teuthology.orchestra.run.smithi117.stderr:adjusted tunables profile to default 2020-11-20T13:55:30.326 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-20T13:55:30.327 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:30.342 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.auLnYu5bfD 2020-11-20T13:55:30.343 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.auLnYu5bfD 2020-11-20T13:55:30.410 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.auLnYu5bfD 2020-11-20T13:55:30.535 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.auLnYu5bfD is 37B 2020-11-20T13:55:30.586 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.auLnYu5bfD 2020-11-20T13:55:30.600 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:30.652 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.mhgDLTuxCn 2020-11-20T13:55:30.653 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.mhgDLTuxCn 2020-11-20T13:55:30.720 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.mhgDLTuxCn 2020-11-20T13:55:30.844 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.mhgDLTuxCn is 37B 2020-11-20T13:55:30.895 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.mhgDLTuxCn 2020-11-20T13:55:30.909 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:30.962 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.MdUT4qEQpV 2020-11-20T13:55:30.963 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.MdUT4qEQpV 2020-11-20T13:55:31.030 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.MdUT4qEQpV 2020-11-20T13:55:31.158 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.MdUT4qEQpV is 37B 2020-11-20T13:55:31.205 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.MdUT4qEQpV 2020-11-20T13:55:31.220 INFO:teuthology.orchestra.run.smithi117:> mktemp 2020-11-20T13:55:31.273 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.HdeXjea9Sa 2020-11-20T13:55:31.274 INFO:teuthology.orchestra.run.smithi117:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.HdeXjea9Sa 2020-11-20T13:55:31.341 INFO:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.HdeXjea9Sa 2020-11-20T13:55:31.464 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.HdeXjea9Sa is 37B 2020-11-20T13:55:31.511 INFO:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.HdeXjea9Sa 2020-11-20T13:55:31.525 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:31.539 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.sQHQi6yvbE 2020-11-20T13:55:31.540 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.sQHQi6yvbE 2020-11-20T13:55:31.608 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.sQHQi6yvbE 2020-11-20T13:55:31.733 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.sQHQi6yvbE is 37B 2020-11-20T13:55:31.781 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.sQHQi6yvbE 2020-11-20T13:55:31.796 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:31.850 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.2Y9eN03WTy 2020-11-20T13:55:31.850 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.2Y9eN03WTy 2020-11-20T13:55:31.918 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.2Y9eN03WTy 2020-11-20T13:55:32.042 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.2Y9eN03WTy is 37B 2020-11-20T13:55:32.090 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.2Y9eN03WTy 2020-11-20T13:55:32.124 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:32.178 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.vPXXyUqJGH 2020-11-20T13:55:32.178 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.vPXXyUqJGH 2020-11-20T13:55:32.246 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.vPXXyUqJGH 2020-11-20T13:55:32.370 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.vPXXyUqJGH is 37B 2020-11-20T13:55:32.417 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.vPXXyUqJGH 2020-11-20T13:55:32.431 INFO:teuthology.orchestra.run.smithi085:> mktemp 2020-11-20T13:55:32.485 INFO:teuthology.orchestra.run.smithi085.stdout:/tmp/tmp.90jZflxi1V 2020-11-20T13:55:32.486 INFO:teuthology.orchestra.run.smithi085:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.90jZflxi1V 2020-11-20T13:55:32.554 INFO:teuthology.orchestra.run.smithi085:> sudo chmod 0666 /tmp/tmp.90jZflxi1V 2020-11-20T13:55:32.678 DEBUG:teuthology.orchestra.remote:smithi085:/tmp/tmp.90jZflxi1V is 37B 2020-11-20T13:55:32.725 INFO:teuthology.orchestra.run.smithi085:> rm -fr /tmp/tmp.90jZflxi1V 2020-11-20T13:55:32.740 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 6425c8e9-c695-4b85-8225-09d3cf8763e7 0 2020-11-20T13:55:32.891 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.887+0000 7f0c4a053700 1 Processor -- start 2020-11-20T13:55:32.892 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.887+0000 7f0c4a053700 1 -- start start 2020-11-20T13:55:32.892 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c4a053700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c440b59a0 0x7f0c44098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:32.892 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c4a053700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c440995e0 con 0x7f0c4405f300 2020-11-20T13:55:32.892 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c4a053700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c44099880 con 0x7f0c44099fc0 2020-11-20T13:55:32.893 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c4a053700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440993c0 con 0x7f0c440b59a0 2020-11-20T13:55:32.893 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c43fff700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f0c4405f300 0x7f0c4409b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:56624/0 (socket says 172.21.15.85:56624) 2020-11-20T13:55:32.893 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c43fff700 1 -- 172.21.15.85:0/1673692675 learned_addr learned my addr 172.21.15.85:0/1673692675 (peer_addr_for_me v1:172.21.15.85:0/0) 2020-11-20T13:55:32.893 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c3bfff700 1 --2- 172.21.15.85:0/1673692675 >> v2:172.21.15.117:3300/0 conn(0x7f0c440b59a0 0x7f0c44098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:32.894 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c437fe700 1 -- 172.21.15.85:0/1673692675 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292132755 0 0) 0x7f0c440995e0 con 0x7f0c4405f300 2020-11-20T13:55:32.894 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.888+0000 7f0c437fe700 1 -- 172.21.15.85:0/1673692675 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c30003230 con 0x7f0c4405f300 2020-11-20T13:55:32.894 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c3bfff700 1 -- 172.21.15.85:0/1673692675 >> v1:172.21.15.117:6789/0 conn(0x7f0c44099fc0 legacy=0x7f0c4409a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:32.894 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c3bfff700 1 -- 172.21.15.85:0/1673692675 >> v1:172.21.15.85:6789/0 conn(0x7f0c4405f300 legacy=0x7f0c4409b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:32.894 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c3bfff700 1 -- 172.21.15.85:0/1673692675 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44099b20 con 0x7f0c440b59a0 2020-11-20T13:55:32.895 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c3bfff700 1 --2- 172.21.15.85:0/1673692675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 0x7f0c44098ed0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f0c2c002f30 tx=0x7f0c2c006dc0).ready entity=mon.1 client_cookie=1a37ed9232bf90f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:32.895 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c437fe700 1 -- 172.21.15.85:0/1673692675 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c2c009800 con 0x7f0c440b59a0 2020-11-20T13:55:32.895 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c437fe700 1 -- 172.21.15.85:0/1673692675 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c009960 con 0x7f0c440b59a0 2020-11-20T13:55:32.895 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c437fe700 1 -- 172.21.15.85:0/1673692675 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c2c009cc0 con 0x7f0c440b59a0 2020-11-20T13:55:32.895 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c4a053700 1 -- 172.21.15.85:0/1673692675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 msgr2=0x7f0c44098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:32.896 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c4a053700 1 --2- 172.21.15.85:0/1673692675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 0x7f0c44098ed0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f0c2c002f30 tx=0x7f0c2c006dc0).stop 2020-11-20T13:55:32.896 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c4a053700 1 -- 172.21.15.85:0/1673692675 shutdown_connections 2020-11-20T13:55:32.896 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.889+0000 7f0c4a053700 1 --2- 172.21.15.85:0/1673692675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 0x7f0c44098ed0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:32.896 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.890+0000 7f0c4a053700 1 -- 172.21.15.85:0/1673692675 >> 172.21.15.85:0/1673692675 conn(0x7f0c440b9490 msgr2=0x7f0c440b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:32.896 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.890+0000 7f0c4a053700 1 -- 172.21.15.85:0/1673692675 shutdown_connections 2020-11-20T13:55:32.897 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.890+0000 7f0c4a053700 1 -- 172.21.15.85:0/1673692675 wait complete. 2020-11-20T13:55:32.899 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.892+0000 7f0c4a053700 1 Processor -- start 2020-11-20T13:55:32.899 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.892+0000 7f0c4a053700 1 -- start start 2020-11-20T13:55:32.900 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c4a053700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c4405f300 0x7f0c4409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:32.900 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c4a053700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c44099fc0 0x7f0c44064260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:32.900 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c4a053700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 0x7f0c440b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:32.900 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c4a053700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440b4950 con 0x7f0c4405f300 2020-11-20T13:55:32.900 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c4a053700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440b25a0 con 0x7f0c440b59a0 2020-11-20T13:55:32.901 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c4a053700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4409b000 con 0x7f0c44099fc0 2020-11-20T13:55:32.901 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c4405f300 0x7f0c4409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:32.901 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c44099fc0 0x7f0c44064260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:32.901 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c43fff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c4405f300 0x7f0c4409aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35688/0 (socket says 172.21.15.85:35688) 2020-11-20T13:55:32.901 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c44099fc0 0x7f0c44064260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58640/0 (socket says 172.21.15.85:58640) 2020-11-20T13:55:32.902 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c43fff700 1 -- 172.21.15.85:0/1416490488 learned_addr learned my addr 172.21.15.85:0/1416490488 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:32.902 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c48df1700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 0x7f0c440b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:32.902 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c43fff700 1 -- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c44099fc0 msgr2=0x7f0c44064260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:32.902 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.893+0000 7f0c43fff700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c44099fc0 0x7f0c44064260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:32.903 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c43fff700 1 -- 172.21.15.85:0/1416490488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 msgr2=0x7f0c440b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:32.903 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c43fff700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 0x7f0c440b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:32.903 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c43fff700 1 -- 172.21.15.85:0/1416490488 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440b2800 con 0x7f0c4405f300 2020-11-20T13:55:32.903 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c3bfff700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c44099fc0 0x7f0c44064260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:32.903 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c43fff700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c4405f300 0x7f0c4409aa10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0c340010d0 tx=0x7f0c340079f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:32.904 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c417fa700 1 -- 172.21.15.85:0/1416490488 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c340072e0 con 0x7f0c4405f300 2020-11-20T13:55:32.904 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c417fa700 1 -- 172.21.15.85:0/1416490488 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34007440 con 0x7f0c4405f300 2020-11-20T13:55:32.904 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c440ae4d0 con 0x7f0c4405f300 2020-11-20T13:55:32.904 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c417fa700 1 -- 172.21.15.85:0/1416490488 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c340057d0 con 0x7f0c4405f300 2020-11-20T13:55:32.904 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c440ae940 con 0x7f0c4405f300 2020-11-20T13:55:32.905 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c417fa700 1 -- 172.21.15.85:0/1416490488 <== mon.0 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f0c34010070 con 0x7f0c4405f300 2020-11-20T13:55:32.905 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.894+0000 7f0c417fa700 1 -- 172.21.15.85:0/1416490488 <== mon.0 v2:172.21.15.85:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f0c340137c0 con 0x7f0c4405f300 2020-11-20T13:55:32.905 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.895+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c440b2990 con 0x7f0c4405f300 2020-11-20T13:55:32.905 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.897+0000 7f0c417fa700 1 -- 172.21.15.85:0/1416490488 <== mon.0 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f0c34031870 con 0x7f0c4405f300 2020-11-20T13:55:32.985 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.984+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6425c8e9-c695-4b85-8225-09d3cf8763e7", "id": 0} v 0) v1 -- 0x7f0c44057050 con 0x7f0c4405f300 2020-11-20T13:55:32.988 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.987+0000 7f0c417fa700 1 -- 172.21.15.85:0/1416490488 <== mon.0 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6425c8e9-c695-4b85-8225-09d3cf8763e7", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c34021070 con 0x7f0c4405f300 2020-11-20T13:55:32.989 INFO:teuthology.orchestra.run.smithi085.stdout:0 2020-11-20T13:55:32.990 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c4405f300 msgr2=0x7f0c4409aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:32.990 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c4405f300 0x7f0c4409aa10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0c340010d0 tx=0x7f0c340079f0).stop 2020-11-20T13:55:32.990 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 shutdown_connections 2020-11-20T13:55:32.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c440b59a0 0x7f0c440b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:32.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c4405f300 0x7f0c4409aa10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:32.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 --2- 172.21.15.85:0/1416490488 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c44099fc0 0x7f0c44064260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:32.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 >> 172.21.15.85:0/1416490488 conn(0x7f0c440b9490 msgr2=0x7f0c4409b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:32.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 shutdown_connections 2020-11-20T13:55:32.992 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:32.988+0000 7f0c4a053700 1 -- 172.21.15.85:0/1416490488 wait complete. 2020-11-20T13:55:32.999 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb 1 2020-11-20T13:55:33.110 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.106+0000 7f53d3237700 1 Processor -- start 2020-11-20T13:55:33.110 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.106+0000 7f53d3237700 1 -- start start 2020-11-20T13:55:33.110 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d3237700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f53cc0b59a0 0x7f53cc098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.111 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d3237700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc0995e0 con 0x7f53cc099fc0 2020-11-20T13:55:33.111 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d3237700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc099880 con 0x7f53cc05f300 2020-11-20T13:55:33.111 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d3237700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53cc0993c0 con 0x7f53cc0b59a0 2020-11-20T13:55:33.111 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d1fd5700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f53cc099fc0 0x7f53cc09a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:56640/0 (socket says 172.21.15.85:56640) 2020-11-20T13:55:33.112 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d1fd5700 1 -- 172.21.15.85:0/3876572893 learned_addr learned my addr 172.21.15.85:0/3876572893 (peer_addr_for_me v1:172.21.15.85:0/0) 2020-11-20T13:55:33.112 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d0fd3700 1 --2- 172.21.15.85:0/3876572893 >> v2:172.21.15.117:3300/0 conn(0x7f53cc0b59a0 0x7f53cc098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.112 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53c3fff700 1 -- 172.21.15.85:0/3876572893 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752176370 0 0) 0x7f53cc0995e0 con 0x7f53cc099fc0 2020-11-20T13:55:33.112 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53c3fff700 1 -- 172.21.15.85:0/3876572893 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b8003230 con 0x7f53cc099fc0 2020-11-20T13:55:33.112 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53c3fff700 1 -- 172.21.15.85:0/3876572893 <== mon.0 v1:172.21.15.85:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2404395124 0 0) 0x7f53b8003230 con 0x7f53cc099fc0 2020-11-20T13:55:33.113 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d0fd3700 1 -- 172.21.15.85:0/3876572893 >> v1:172.21.15.117:6789/0 conn(0x7f53cc05f300 legacy=0x7f53cc09b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.113 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d0fd3700 1 -- 172.21.15.85:0/3876572893 >> v1:172.21.15.85:6789/0 conn(0x7f53cc099fc0 legacy=0x7f53cc09a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.113 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d0fd3700 1 -- 172.21.15.85:0/3876572893 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc099b20 con 0x7f53cc0b59a0 2020-11-20T13:55:33.113 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.107+0000 7f53d0fd3700 1 --2- 172.21.15.85:0/3876572893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc0b59a0 0x7f53cc098ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f53c4001bf0 tx=0x7f53bc006dc0).ready entity=mon.1 client_cookie=d53ed8164cc1653b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.114 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53c3fff700 1 -- 172.21.15.85:0/3876572893 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c4001a50 con 0x7f53cc0b59a0 2020-11-20T13:55:33.114 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53c3fff700 1 -- 172.21.15.85:0/3876572893 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc009670 con 0x7f53cc0b59a0 2020-11-20T13:55:33.114 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53c3fff700 1 -- 172.21.15.85:0/3876572893 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53bc0097e0 con 0x7f53cc0b59a0 2020-11-20T13:55:33.114 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53d3237700 1 -- 172.21.15.85:0/3876572893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc0b59a0 msgr2=0x7f53cc098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.114 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53d3237700 1 --2- 172.21.15.85:0/3876572893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc0b59a0 0x7f53cc098ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f53c4001bf0 tx=0x7f53bc006dc0).stop 2020-11-20T13:55:33.114 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53d3237700 1 -- 172.21.15.85:0/3876572893 shutdown_connections 2020-11-20T13:55:33.115 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53d3237700 1 --2- 172.21.15.85:0/3876572893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc0b59a0 0x7f53cc098ed0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.115 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53d3237700 1 -- 172.21.15.85:0/3876572893 >> 172.21.15.85:0/3876572893 conn(0x7f53cc0b9490 msgr2=0x7f53cc0b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.115 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53d3237700 1 -- 172.21.15.85:0/3876572893 shutdown_connections 2020-11-20T13:55:33.115 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.108+0000 7f53d3237700 1 -- 172.21.15.85:0/3876572893 wait complete. 2020-11-20T13:55:33.118 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.109+0000 7f53d3237700 1 Processor -- start 2020-11-20T13:55:33.118 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.109+0000 7f53d3237700 1 -- start start 2020-11-20T13:55:33.119 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d3237700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f53cc05f300 0x7f53cc063780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.119 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d3237700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc099fc0 0x7f53cc08f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.119 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d3237700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f53cc0b59a0 0x7f53cc090680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.119 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d3237700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0902b0 con 0x7f53cc05f300 2020-11-20T13:55:33.120 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d3237700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc090010 con 0x7f53cc099fc0 2020-11-20T13:55:33.120 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d3237700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0926c0 con 0x7f53cc0b59a0 2020-11-20T13:55:33.120 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d17d4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f53cc05f300 0x7f53cc063780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.120 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d1fd5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f53cc0b59a0 0x7f53cc090680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.120 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d17d4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f53cc05f300 0x7f53cc063780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35700/0 (socket says 172.21.15.85:35700) 2020-11-20T13:55:33.121 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d17d4700 1 -- 172.21.15.85:0/1060249694 learned_addr learned my addr 172.21.15.85:0/1060249694 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:33.121 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.110+0000 7f53d0fd3700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc099fc0 0x7f53cc08f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.121 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d17d4700 1 -- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f53cc0b59a0 msgr2=0x7f53cc090680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.121 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d17d4700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f53cc0b59a0 0x7f53cc090680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.122 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d17d4700 1 -- 172.21.15.85:0/1060249694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc099fc0 msgr2=0x7f53cc08f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.122 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d17d4700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc099fc0 0x7f53cc08f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.122 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d17d4700 1 -- 172.21.15.85:0/1060249694 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc144da0 con 0x7f53cc05f300 2020-11-20T13:55:33.122 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d1fd5700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f53cc0b59a0 0x7f53cc090680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.122 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d17d4700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f53cc05f300 0x7f53cc063780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f53cc062180 tx=0x7f53c8007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.123 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53c1ffb700 1 -- 172.21.15.85:0/1060249694 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c8007030 con 0x7f53cc05f300 2020-11-20T13:55:33.123 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53c1ffb700 1 -- 172.21.15.85:0/1060249694 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8007190 con 0x7f53cc05f300 2020-11-20T13:55:33.123 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc063c70 con 0x7f53cc05f300 2020-11-20T13:55:33.123 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53c1ffb700 1 -- 172.21.15.85:0/1060249694 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c80064e0 con 0x7f53cc05f300 2020-11-20T13:55:33.123 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc066b70 con 0x7f53cc05f300 2020-11-20T13:55:33.124 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53c1ffb700 1 -- 172.21.15.85:0/1060249694 <== mon.0 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f53c8010070 con 0x7f53cc05f300 2020-11-20T13:55:33.124 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.111+0000 7f53c1ffb700 1 -- 172.21.15.85:0/1060249694 <== mon.0 v2:172.21.15.85:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f53c8006dd0 con 0x7f53cc05f300 2020-11-20T13:55:33.124 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.112+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc057050 con 0x7f53cc05f300 2020-11-20T13:55:33.124 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.116+0000 7f53c1ffb700 1 -- 172.21.15.85:0/1060249694 <== mon.0 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f53c8031870 con 0x7f53cc05f300 2020-11-20T13:55:33.203 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.201+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb", "id": 1} v 0) v1 -- 0x7f53cc05ca00 con 0x7f53cc05f300 2020-11-20T13:55:33.205 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.204+0000 7f53c1ffb700 1 -- 172.21.15.85:0/1060249694 <== mon.0 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f53c8021070 con 0x7f53cc05f300 2020-11-20T13:55:33.206 INFO:teuthology.orchestra.run.smithi085.stdout:1 2020-11-20T13:55:33.207 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f53cc05f300 msgr2=0x7f53cc063780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.207 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f53cc05f300 0x7f53cc063780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f53cc062180 tx=0x7f53c8007b50).stop 2020-11-20T13:55:33.207 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 shutdown_connections 2020-11-20T13:55:33.207 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f53cc0b59a0 0x7f53cc090680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f53cc05f300 0x7f53cc063780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 --2- 172.21.15.85:0/1060249694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53cc099fc0 0x7f53cc08f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 >> 172.21.15.85:0/1060249694 conn(0x7f53cc0b9490 msgr2=0x7f53cc07fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 shutdown_connections 2020-11-20T13:55:33.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.205+0000 7f53d3237700 1 -- 172.21.15.85:0/1060249694 wait complete. 2020-11-20T13:55:33.216 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 2aaa1ba8-9363-4263-b86b-b163296df2dc 2 2020-11-20T13:55:33.329 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.325+0000 7f8b24ccb700 1 Processor -- start 2020-11-20T13:55:33.329 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.325+0000 7f8b24ccb700 1 -- start start 2020-11-20T13:55:33.329 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.325+0000 7f8b24ccb700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8b2005f300 0x7f8b20098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.330 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.325+0000 7f8b24ccb700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8b20099fc0 0x7f8b2009a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.330 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.325+0000 7f8b24ccb700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b200995e0 con 0x7f8b200b59a0 2020-11-20T13:55:33.330 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.325+0000 7f8b24ccb700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b200993c0 con 0x7f8b20099fc0 2020-11-20T13:55:33.330 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.325+0000 7f8b24ccb700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b20099880 con 0x7f8b2005f300 2020-11-20T13:55:33.330 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1f59e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8b20099fc0 0x7f8b2009a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.331 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1e59c700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f8b200b59a0 0x7f8b2009b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:56648/0 (socket says 172.21.15.85:56648) 2020-11-20T13:55:33.331 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1f59e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8b20099fc0 0x7f8b2009a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35710/0 (socket says 172.21.15.85:35710) 2020-11-20T13:55:33.331 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1e59c700 1 -- 172.21.15.85:0/1650172898 learned_addr learned my addr 172.21.15.85:0/1650172898 (peer_addr_for_me v1:172.21.15.85:0/0) 2020-11-20T13:55:33.331 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1ed9d700 1 --2- 172.21.15.85:0/1650172898 >> v2:172.21.15.117:3300/0 conn(0x7f8b2005f300 0x7f8b20098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.332 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1f59e700 1 -- 172.21.15.85:0/1650172898 >> v2:172.21.15.117:3300/0 conn(0x7f8b2005f300 msgr2=0x7f8b20098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:33.332 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1f59e700 1 --2- 172.21.15.85:0/1650172898 >> v2:172.21.15.117:3300/0 conn(0x7f8b2005f300 0x7f8b20098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.332 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1f59e700 1 -- 172.21.15.85:0/1650172898 >> v1:172.21.15.85:6789/0 conn(0x7f8b200b59a0 legacy=0x7f8b2009b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.332 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1f59e700 1 -- 172.21.15.85:0/1650172898 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20099b20 con 0x7f8b20099fc0 2020-11-20T13:55:33.332 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1dd9b700 1 -- 172.21.15.85:0/1650172898 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842692083 0 0) 0x7f8b200995e0 con 0x7f8b200b59a0 2020-11-20T13:55:33.333 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1f59e700 1 --2- 172.21.15.85:0/1650172898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b20099fc0 0x7f8b2009a410 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f8b180032b0 tx=0x7f8b180091a0).ready entity=mon.0 client_cookie=bbec0f6d9e248dc0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.333 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1dd9b700 1 -- 172.21.15.85:0/1650172898 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b1800b9a0 con 0x7f8b20099fc0 2020-11-20T13:55:33.333 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1dd9b700 1 -- 172.21.15.85:0/1650172898 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1800bb00 con 0x7f8b20099fc0 2020-11-20T13:55:33.333 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b1dd9b700 1 -- 172.21.15.85:0/1650172898 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b1800bc70 con 0x7f8b20099fc0 2020-11-20T13:55:33.334 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b24ccb700 1 -- 172.21.15.85:0/1650172898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b20099fc0 msgr2=0x7f8b2009a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.334 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.326+0000 7f8b24ccb700 1 --2- 172.21.15.85:0/1650172898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b20099fc0 0x7f8b2009a410 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f8b180032b0 tx=0x7f8b180091a0).stop 2020-11-20T13:55:33.334 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.327+0000 7f8b24ccb700 1 -- 172.21.15.85:0/1650172898 shutdown_connections 2020-11-20T13:55:33.334 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.327+0000 7f8b24ccb700 1 --2- 172.21.15.85:0/1650172898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b20099fc0 0x7f8b2009a410 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.335 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.327+0000 7f8b24ccb700 1 --2- 172.21.15.85:0/1650172898 >> v2:172.21.15.117:3300/0 conn(0x7f8b2005f300 0x7f8b20098ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.335 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.327+0000 7f8b24ccb700 1 -- 172.21.15.85:0/1650172898 >> 172.21.15.85:0/1650172898 conn(0x7f8b200b9490 msgr2=0x7f8b200b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.335 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.327+0000 7f8b24ccb700 1 -- 172.21.15.85:0/1650172898 shutdown_connections 2020-11-20T13:55:33.336 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.327+0000 7f8b24ccb700 1 -- 172.21.15.85:0/1650172898 wait complete. 2020-11-20T13:55:33.338 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.328+0000 7f8b24ccb700 1 Processor -- start 2020-11-20T13:55:33.339 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.328+0000 7f8b24ccb700 1 -- start start 2020-11-20T13:55:33.339 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2005f300 0x7f8b200b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.339 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8b20099fc0 0x7f8b20145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.339 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b200b59a0 0x7f8b20145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.340 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200b2270 con 0x7f8b200b59a0 2020-11-20T13:55:33.340 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200b49b0 con 0x7f8b2005f300 2020-11-20T13:55:33.340 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200b4c50 con 0x7f8b20099fc0 2020-11-20T13:55:33.340 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1f59e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b200b59a0 0x7f8b20145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.340 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8b20099fc0 0x7f8b20145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.341 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8b20099fc0 0x7f8b20145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58666/0 (socket says 172.21.15.85:58666) 2020-11-20T13:55:33.341 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1f59e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b200b59a0 0x7f8b20145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35716/0 (socket says 172.21.15.85:35716) 2020-11-20T13:55:33.341 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 -- 172.21.15.85:0/3778693028 learned_addr learned my addr 172.21.15.85:0/3778693028 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:33.341 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1ed9d700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2005f300 0x7f8b200b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.341 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 -- 172.21.15.85:0/3778693028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2005f300 msgr2=0x7f8b200b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.342 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2005f300 0x7f8b200b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.342 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 -- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b200b59a0 msgr2=0x7f8b20145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.342 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b200b59a0 0x7f8b20145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.342 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 -- 172.21.15.85:0/3778693028 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200b4ef0 con 0x7f8b20099fc0 2020-11-20T13:55:33.343 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1f59e700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b200b59a0 0x7f8b20145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.343 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b1e59c700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8b20099fc0 0x7f8b20145610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8b10000e90 tx=0x7f8b10007a90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.343 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b0f7fe700 1 -- 172.21.15.85:0/3778693028 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b100072e0 con 0x7f8b20099fc0 2020-11-20T13:55:33.343 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b0f7fe700 1 -- 172.21.15.85:0/3778693028 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10007440 con 0x7f8b20099fc0 2020-11-20T13:55:33.343 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b200b2c30 con 0x7f8b20099fc0 2020-11-20T13:55:33.344 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b0f7fe700 1 -- 172.21.15.85:0/3778693028 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b100057d0 con 0x7f8b20099fc0 2020-11-20T13:55:33.344 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.329+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b200b4470 con 0x7f8b20099fc0 2020-11-20T13:55:33.344 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.330+0000 7f8b0f7fe700 1 -- 172.21.15.85:0/3778693028 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f8b10010070 con 0x7f8b20099fc0 2020-11-20T13:55:33.344 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.330+0000 7f8b0f7fe700 1 -- 172.21.15.85:0/3778693028 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f8b100138a0 con 0x7f8b20099fc0 2020-11-20T13:55:33.344 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.331+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b20057050 con 0x7f8b20099fc0 2020-11-20T13:55:33.345 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.333+0000 7f8b0f7fe700 1 -- 172.21.15.85:0/3778693028 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f8b10013450 con 0x7f8b20099fc0 2020-11-20T13:55:33.419 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.418+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2aaa1ba8-9363-4263-b86b-b163296df2dc", "id": 2} v 0) v1 -- 0x7f8b2009c150 con 0x7f8b20099fc0 2020-11-20T13:55:33.423 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.422+0000 7f8b0f7fe700 1 -- 172.21.15.85:0/3778693028 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2aaa1ba8-9363-4263-b86b-b163296df2dc", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8b10013670 con 0x7f8b20099fc0 2020-11-20T13:55:33.423 INFO:teuthology.orchestra.run.smithi085.stdout:2 2020-11-20T13:55:33.424 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8b20099fc0 msgr2=0x7f8b20145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8b20099fc0 0x7f8b20145610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8b10000e90 tx=0x7f8b10007a90).stop 2020-11-20T13:55:33.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 shutdown_connections 2020-11-20T13:55:33.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8b200b59a0 0x7f8b20145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2005f300 0x7f8b200b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 --2- 172.21.15.85:0/3778693028 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8b20099fc0 0x7f8b20145610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 >> 172.21.15.85:0/3778693028 conn(0x7f8b200b9490 msgr2=0x7f8b20065a40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 shutdown_connections 2020-11-20T13:55:33.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.423+0000 7f8b24ccb700 1 -- 172.21.15.85:0/3778693028 wait complete. 2020-11-20T13:55:33.434 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 61687ded-23a5-4350-aa98-f5be6d396d11 3 2020-11-20T13:55:33.545 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.540+0000 7f209430c700 1 Processor -- start 2020-11-20T13:55:33.545 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.540+0000 7f209430c700 1 -- start start 2020-11-20T13:55:33.546 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f209430c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f208c05f300 0x7f208c098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.546 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f209430c700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c0995e0 con 0x7f208c0b59a0 2020-11-20T13:55:33.546 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f209430c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c099880 con 0x7f208c099fc0 2020-11-20T13:55:33.546 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f209430c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c0993c0 con 0x7f208c05f300 2020-11-20T13:55:33.547 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20920a8700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f208c0b59a0 0x7f208c09b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:56662/0 (socket says 172.21.15.85:56662) 2020-11-20T13:55:33.547 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20920a8700 1 -- 172.21.15.85:0/1811599054 learned_addr learned my addr 172.21.15.85:0/1811599054 (peer_addr_for_me v1:172.21.15.85:0/0) 2020-11-20T13:55:33.547 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20928a9700 1 --2- 172.21.15.85:0/1811599054 >> v2:172.21.15.117:3300/0 conn(0x7f208c05f300 0x7f208c098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.547 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286090915 0 0) 0x7f208c0995e0 con 0x7f208c0b59a0 2020-11-20T13:55:33.547 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2078003230 con 0x7f208c0b59a0 2020-11-20T13:55:33.548 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643458697 0 0) 0x7f208c099880 con 0x7f208c099fc0 2020-11-20T13:55:33.548 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f208c0995e0 con 0x7f208c099fc0 2020-11-20T13:55:33.548 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 <== mon.0 v1:172.21.15.85:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2257994401 0 0) 0x7f2078003230 con 0x7f208c0b59a0 2020-11-20T13:55:33.548 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 >> v2:172.21.15.117:3300/0 conn(0x7f208c05f300 msgr2=0x7f208c098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:33.548 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 --2- 172.21.15.85:0/1811599054 >> v2:172.21.15.117:3300/0 conn(0x7f208c05f300 0x7f208c098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.549 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 >> v1:172.21.15.117:6789/0 conn(0x7f208c099fc0 legacy=0x7f208c09a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.549 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 --> v1:172.21.15.85:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c099b20 con 0x7f208c0b59a0 2020-11-20T13:55:33.549 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 <== mon.0 v1:172.21.15.85:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f207c0016d0 con 0x7f208c0b59a0 2020-11-20T13:55:33.549 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 >> v1:172.21.15.85:6789/0 conn(0x7f208c0b59a0 legacy=0x7f208c09b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.549 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f20780038b0 0x7f2078003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.550 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20780044d0 0x7f2078004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.550 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2078004e40 0x7f207804f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.550 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0993c0 con 0x7f2078004e40 2020-11-20T13:55:33.550 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207804fc20 con 0x7f20780044d0 2020-11-20T13:55:33.550 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207804fdb0 con 0x7f20780038b0 2020-11-20T13:55:33.551 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20928a9700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f20780038b0 0x7f2078003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.551 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20930aa700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2078004e40 0x7f207804f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.551 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.541+0000 7f20920a8700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20780044d0 0x7f2078004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.551 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20930aa700 1 -- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f20780038b0 msgr2=0x7f2078003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:33.552 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20930aa700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f20780038b0 0x7f2078003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.552 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20930aa700 1 -- 172.21.15.85:0/1811599054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20780044d0 msgr2=0x7f2078004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:33.552 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20930aa700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20780044d0 0x7f2078004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.552 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20930aa700 1 -- 172.21.15.85:0/1811599054 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078003230 con 0x7f2078004e40 2020-11-20T13:55:33.552 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20930aa700 1 -- 172.21.15.85:0/1811599054 reap_dead start 2020-11-20T13:55:33.553 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20930aa700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2078004e40 0x7f207804f730 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f2084001450 tx=0x7f2084007e50).ready entity=mon.0 client_cookie=852d844e49de7cb2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.553 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f208400cc00 con 0x7f2078004e40 2020-11-20T13:55:33.553 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208400cd60 con 0x7f2078004e40 2020-11-20T13:55:33.553 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f20918a7700 1 -- 172.21.15.85:0/1811599054 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20840125e0 con 0x7f2078004e40 2020-11-20T13:55:33.553 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f209430c700 1 -- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2078004e40 msgr2=0x7f207804f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.554 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.542+0000 7f209430c700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2078004e40 0x7f207804f730 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f2084001450 tx=0x7f2084007e50).stop 2020-11-20T13:55:33.554 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.543+0000 7f209430c700 1 -- 172.21.15.85:0/1811599054 shutdown_connections 2020-11-20T13:55:33.554 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.543+0000 7f209430c700 1 --2- 172.21.15.85:0/1811599054 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2078004e40 0x7f207804f730 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.554 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.543+0000 7f209430c700 1 -- 172.21.15.85:0/1811599054 >> 172.21.15.85:0/1811599054 conn(0x7f208c0b9490 msgr2=0x7f208c0b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.554 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.543+0000 7f209430c700 1 -- 172.21.15.85:0/1811599054 shutdown_connections 2020-11-20T13:55:33.555 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.543+0000 7f209430c700 1 -- 172.21.15.85:0/1811599054 wait complete. 2020-11-20T13:55:33.555 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.544+0000 7f209430c700 1 Processor -- start 2020-11-20T13:55:33.555 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.544+0000 7f209430c700 1 -- start start 2020-11-20T13:55:33.557 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.544+0000 7f209430c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2078004e40 0x7f208c145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.557 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.544+0000 7f209430c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f208c0b3970 0x7f208c0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.558 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.544+0000 7f209430c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f208c0aee40 0x7f208c0afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.558 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.544+0000 7f209430c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c145510 con 0x7f208c0aee40 2020-11-20T13:55:33.558 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f209430c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c1457b0 con 0x7f2078004e40 2020-11-20T13:55:33.558 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f209430c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c090260 con 0x7f208c0b3970 2020-11-20T13:55:33.558 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f208c0aee40 0x7f208c0afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20920a8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f208c0b3970 0x7f208c0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20920a8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f208c0b3970 0x7f208c0ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58684/0 (socket says 172.21.15.85:58684) 2020-11-20T13:55:33.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f208c0aee40 0x7f208c0afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35734/0 (socket says 172.21.15.85:35734) 2020-11-20T13:55:33.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20920a8700 1 -- 172.21.15.85:0/3476929432 learned_addr learned my addr 172.21.15.85:0/3476929432 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:33.559 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20928a9700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2078004e40 0x7f208c145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.560 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 -- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f208c0b3970 msgr2=0x7f208c0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.560 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f208c0b3970 0x7f208c0ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.560 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 -- 172.21.15.85:0/3476929432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2078004e40 msgr2=0x7f208c145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.560 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2078004e40 0x7f208c145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.561 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 -- 172.21.15.85:0/3476929432 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c146140 con 0x7f208c0aee40 2020-11-20T13:55:33.561 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20920a8700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f208c0b3970 0x7f208c0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.561 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20928a9700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2078004e40 0x7f208c145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.561 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20930aa700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f208c0aee40 0x7f208c0afd60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2084001450 tx=0x7f2084014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.562 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20837fe700 1 -- 172.21.15.85:0/3476929432 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2084007b80 con 0x7f208c0aee40 2020-11-20T13:55:33.562 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20837fe700 1 -- 172.21.15.85:0/3476929432 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208400ced0 con 0x7f208c0aee40 2020-11-20T13:55:33.562 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208c0b32d0 con 0x7f208c0aee40 2020-11-20T13:55:33.562 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20837fe700 1 -- 172.21.15.85:0/3476929432 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f208401e670 con 0x7f208c0aee40 2020-11-20T13:55:33.562 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c156c20 con 0x7f208c0aee40 2020-11-20T13:55:33.563 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20837fe700 1 -- 172.21.15.85:0/3476929432 <== mon.0 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2084025030 con 0x7f208c0aee40 2020-11-20T13:55:33.563 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.545+0000 7f20837fe700 1 -- 172.21.15.85:0/3476929432 <== mon.0 v2:172.21.15.85:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f2084027870 con 0x7f208c0aee40 2020-11-20T13:55:33.563 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.546+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c0b3d50 con 0x7f208c0aee40 2020-11-20T13:55:33.563 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.548+0000 7f20837fe700 1 -- 172.21.15.85:0/3476929432 <== mon.0 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f208c0b3d50 con 0x7f208c0aee40 2020-11-20T13:55:33.636 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.635+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "61687ded-23a5-4350-aa98-f5be6d396d11", "id": 3} v 0) v1 -- 0x7f208c1462d0 con 0x7f208c0aee40 2020-11-20T13:55:33.639 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.638+0000 7f20837fe700 1 -- 172.21.15.85:0/3476929432 <== mon.0 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "61687ded-23a5-4350-aa98-f5be6d396d11", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2084023070 con 0x7f208c0aee40 2020-11-20T13:55:33.639 INFO:teuthology.orchestra.run.smithi085.stdout:3 2020-11-20T13:55:33.640 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f208c0aee40 msgr2=0x7f208c0afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.640 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f208c0aee40 0x7f208c0afd60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2084001450 tx=0x7f2084014000).stop 2020-11-20T13:55:33.641 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 shutdown_connections 2020-11-20T13:55:33.641 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f208c0aee40 0x7f208c0afd60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.641 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2078004e40 0x7f208c145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.641 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 --2- 172.21.15.85:0/3476929432 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f208c0b3970 0x7f208c0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.642 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 >> 172.21.15.85:0/3476929432 conn(0x7f208c0b9490 msgr2=0x7f208c09b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.642 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 shutdown_connections 2020-11-20T13:55:33.642 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.639+0000 7f209430c700 1 -- 172.21.15.85:0/3476929432 wait complete. 2020-11-20T13:55:33.650 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 48d54979-1a08-4759-846e-b56eb855d218 4 2020-11-20T13:55:33.760 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.756+0000 7f3951295700 1 Processor -- start 2020-11-20T13:55:33.760 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.756+0000 7f3951295700 1 -- start start 2020-11-20T13:55:33.760 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f3951295700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f394c0b59a0 0x7f394c098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.760 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f3951295700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f394c099fc0 0x7f394c09a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.761 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f3951295700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c0995e0 con 0x7f394c05f300 2020-11-20T13:55:33.761 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f3951295700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c0993c0 con 0x7f394c0b59a0 2020-11-20T13:55:33.761 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f3951295700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c099880 con 0x7f394c099fc0 2020-11-20T13:55:33.761 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f394c0b59a0 0x7f394c098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.762 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f394c0b59a0 0x7f394c098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35740/0 (socket says 172.21.15.85:35740) 2020-11-20T13:55:33.762 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 -- 172.21.15.85:0/1847623884 learned_addr learned my addr 172.21.15.85:0/1847623884 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:33.762 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394bfff700 1 --2- 172.21.15.85:0/1847623884 >> v2:172.21.15.117:3300/0 conn(0x7f394c099fc0 0x7f394c09a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.762 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 -- 172.21.15.85:0/1847623884 >> v2:172.21.15.117:3300/0 conn(0x7f394c099fc0 msgr2=0x7f394c09a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:33.762 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 --2- 172.21.15.85:0/1847623884 >> v2:172.21.15.117:3300/0 conn(0x7f394c099fc0 0x7f394c09a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.763 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 -- 172.21.15.85:0/1847623884 >> v1:172.21.15.117:6789/0 conn(0x7f394c05f300 legacy=0x7f394c09b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.763 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 -- 172.21.15.85:0/1847623884 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c099b20 con 0x7f394c0b59a0 2020-11-20T13:55:33.763 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394affd700 1 --2- 172.21.15.85:0/1847623884 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c098ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f393c002f30 tx=0x7f393c006e80).ready entity=mon.0 client_cookie=d42cf75057a6e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.763 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394a7fc700 1 -- 172.21.15.85:0/1847623884 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393c0097f0 con 0x7f394c0b59a0 2020-11-20T13:55:33.764 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394a7fc700 1 -- 172.21.15.85:0/1847623884 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c009950 con 0x7f394c0b59a0 2020-11-20T13:55:33.764 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f394a7fc700 1 -- 172.21.15.85:0/1847623884 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393c009cb0 con 0x7f394c0b59a0 2020-11-20T13:55:33.764 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.757+0000 7f3951295700 1 -- 172.21.15.85:0/1847623884 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 msgr2=0x7f394c098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.764 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.758+0000 7f3951295700 1 --2- 172.21.15.85:0/1847623884 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c098ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f393c002f30 tx=0x7f393c006e80).stop 2020-11-20T13:55:33.764 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.758+0000 7f3951295700 1 -- 172.21.15.85:0/1847623884 shutdown_connections 2020-11-20T13:55:33.765 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.758+0000 7f3951295700 1 --2- 172.21.15.85:0/1847623884 >> v2:172.21.15.117:3300/0 conn(0x7f394c099fc0 0x7f394c09a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.765 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.758+0000 7f3951295700 1 --2- 172.21.15.85:0/1847623884 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c098ed0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.765 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.758+0000 7f3951295700 1 -- 172.21.15.85:0/1847623884 >> 172.21.15.85:0/1847623884 conn(0x7f394c0b9490 msgr2=0x7f394c0b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.765 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.758+0000 7f3951295700 1 -- 172.21.15.85:0/1847623884 shutdown_connections 2020-11-20T13:55:33.765 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.758+0000 7f3951295700 1 -- 172.21.15.85:0/1847623884 wait complete. 2020-11-20T13:55:33.768 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.759+0000 7f3951295700 1 Processor -- start 2020-11-20T13:55:33.768 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.759+0000 7f3951295700 1 -- start start 2020-11-20T13:55:33.768 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f3951295700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f394c05f300 0x7f394c141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.768 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f3951295700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c099fc0 0x7f394c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.769 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f3951295700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c1453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.769 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f3951295700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0b1da0 con 0x7f394c0b59a0 2020-11-20T13:55:33.769 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f3951295700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0b1170 con 0x7f394c099fc0 2020-11-20T13:55:33.769 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f3951295700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0b1410 con 0x7f394c05f300 2020-11-20T13:55:33.769 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f394b7fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f394c05f300 0x7f394c141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.770 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f394bfff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c1453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.770 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f394b7fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f394c05f300 0x7f394c141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58694/0 (socket says 172.21.15.85:58694) 2020-11-20T13:55:33.770 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f394b7fe700 1 -- 172.21.15.85:0/3931997176 learned_addr learned my addr 172.21.15.85:0/3931997176 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:33.770 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.760+0000 7f394affd700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c099fc0 0x7f394c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.770 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394b7fe700 1 -- 172.21.15.85:0/3931997176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c099fc0 msgr2=0x7f394c0b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.771 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394b7fe700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c099fc0 0x7f394c0b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.771 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394b7fe700 1 -- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 msgr2=0x7f394c1453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.771 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394b7fe700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c1453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.771 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394b7fe700 1 -- 172.21.15.85:0/3931997176 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c0b49c0 con 0x7f394c05f300 2020-11-20T13:55:33.772 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394bfff700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c1453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.772 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394affd700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c099fc0 0x7f394c0b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.772 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f394b7fe700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f394c05f300 0x7f394c141870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f394c09d020 tx=0x7f3940007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.772 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f3933fff700 1 -- 172.21.15.85:0/3931997176 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f394000f470 con 0x7f394c05f300 2020-11-20T13:55:33.772 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f3933fff700 1 -- 172.21.15.85:0/3931997176 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000f5d0 con 0x7f394c05f300 2020-11-20T13:55:33.772 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f3933fff700 1 -- 172.21.15.85:0/3931997176 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f394000f930 con 0x7f394c05f300 2020-11-20T13:55:33.773 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c0b4bb0 con 0x7f394c05f300 2020-11-20T13:55:33.773 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c145a60 con 0x7f394c05f300 2020-11-20T13:55:33.773 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f3933fff700 1 -- 172.21.15.85:0/3931997176 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f3940019030 con 0x7f394c05f300 2020-11-20T13:55:33.773 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.761+0000 7f3933fff700 1 -- 172.21.15.85:0/3931997176 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f394001b8e0 con 0x7f394c05f300 2020-11-20T13:55:33.774 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.762+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c0b0fb0 con 0x7f394c05f300 2020-11-20T13:55:33.774 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.764+0000 7f3933fff700 1 -- 172.21.15.85:0/3931997176 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f3940017070 con 0x7f394c05f300 2020-11-20T13:55:33.852 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.851+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "48d54979-1a08-4759-846e-b56eb855d218", "id": 4} v 0) v1 -- 0x7f394c0b15d0 con 0x7f394c05f300 2020-11-20T13:55:33.855 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.854+0000 7f3933fff700 1 -- 172.21.15.85:0/3931997176 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "48d54979-1a08-4759-846e-b56eb855d218", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f394001bc40 con 0x7f394c05f300 2020-11-20T13:55:33.855 INFO:teuthology.orchestra.run.smithi085.stdout:4 2020-11-20T13:55:33.857 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f394c05f300 msgr2=0x7f394c141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.857 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f394c05f300 0x7f394c141870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f394c09d020 tx=0x7f3940007b90).stop 2020-11-20T13:55:33.857 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 shutdown_connections 2020-11-20T13:55:33.857 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f394c05f300 0x7f394c141870 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.857 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f394c0b59a0 0x7f394c1453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.858 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 --2- 172.21.15.85:0/3931997176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f394c099fc0 0x7f394c0b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.858 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 >> 172.21.15.85:0/3931997176 conn(0x7f394c0b9490 msgr2=0x7f394c066890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.858 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.855+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 shutdown_connections 2020-11-20T13:55:33.858 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.856+0000 7f3951295700 1 -- 172.21.15.85:0/3931997176 wait complete. 2020-11-20T13:55:33.867 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 8e801f28-c346-40d6-a3f8-5e4d072ed72d 5 2020-11-20T13:55:33.978 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.974+0000 7f0e442f5700 1 Processor -- start 2020-11-20T13:55:33.978 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.974+0000 7f0e442f5700 1 -- start start 2020-11-20T13:55:33.979 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e442f5700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f0e3c0b59a0 0x7f0e3c098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.979 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e442f5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.979 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e442f5700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c0995e0 con 0x7f0e3c05f300 2020-11-20T13:55:33.979 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e442f5700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c0993c0 con 0x7f0e3c0b59a0 2020-11-20T13:55:33.980 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e442f5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c099880 con 0x7f0e3c099fc0 2020-11-20T13:55:33.980 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f0e3c0b59a0 0x7f0e3c098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.980 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e43093700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.980 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f0e3c0b59a0 0x7f0e3c098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35750/0 (socket says 172.21.15.85:35750) 2020-11-20T13:55:33.980 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 -- 172.21.15.85:0/3378150933 learned_addr learned my addr 172.21.15.85:0/3378150933 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:33.981 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42892700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f0e3c05f300 0x7f0e3c09b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:56690/0 (socket says 172.21.15.85:56690) 2020-11-20T13:55:33.981 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 -- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 msgr2=0x7f0e3c09a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:33.981 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 --2- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.981 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 -- 172.21.15.85:0/3378150933 >> v1:172.21.15.85:6789/0 conn(0x7f0e3c05f300 legacy=0x7f0e3c09b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.981 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e41890700 1 -- 172.21.15.85:0/3378150933 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275295100 0 0) 0x7f0e3c0995e0 con 0x7f0e3c05f300 2020-11-20T13:55:33.982 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 -- 172.21.15.85:0/3378150933 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c099b20 con 0x7f0e3c0b59a0 2020-11-20T13:55:33.982 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e43093700 1 --2- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.982 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e42091700 1 --2- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c098ed0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f0e38002f30 tx=0x7f0e38006e80).ready entity=mon.0 client_cookie=680089a9569af8c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.982 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e41890700 1 -- 172.21.15.85:0/3378150933 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e380097f0 con 0x7f0e3c0b59a0 2020-11-20T13:55:33.982 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e41890700 1 -- 172.21.15.85:0/3378150933 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38009950 con 0x7f0e3c0b59a0 2020-11-20T13:55:33.983 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e41890700 1 -- 172.21.15.85:0/3378150933 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e38009cb0 con 0x7f0e3c0b59a0 2020-11-20T13:55:33.983 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e442f5700 1 -- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 msgr2=0x7f0e3c098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.983 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.975+0000 7f0e442f5700 1 --2- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c098ed0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f0e38002f30 tx=0x7f0e38006e80).stop 2020-11-20T13:55:33.983 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.976+0000 7f0e442f5700 1 -- 172.21.15.85:0/3378150933 shutdown_connections 2020-11-20T13:55:33.983 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.976+0000 7f0e442f5700 1 --2- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.984 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.976+0000 7f0e442f5700 1 --2- 172.21.15.85:0/3378150933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c098ed0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.984 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.976+0000 7f0e442f5700 1 -- 172.21.15.85:0/3378150933 >> 172.21.15.85:0/3378150933 conn(0x7f0e3c0b9490 msgr2=0x7f0e3c0b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:33.984 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.976+0000 7f0e442f5700 1 -- 172.21.15.85:0/3378150933 shutdown_connections 2020-11-20T13:55:33.984 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.976+0000 7f0e442f5700 1 -- 172.21.15.85:0/3378150933 wait complete. 2020-11-20T13:55:33.984 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.977+0000 7f0e442f5700 1 Processor -- start 2020-11-20T13:55:33.985 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.977+0000 7f0e442f5700 1 -- start start 2020-11-20T13:55:33.988 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e442f5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c05f300 0x7f0e3c090e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.989 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e442f5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.989 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e442f5700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c0b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:33.989 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e442f5700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c090180 con 0x7f0e3c0b59a0 2020-11-20T13:55:33.989 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e442f5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c090420 con 0x7f0e3c05f300 2020-11-20T13:55:33.990 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e442f5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0906c0 con 0x7f0e3c099fc0 2020-11-20T13:55:33.990 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.990 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e43093700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c0b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.990 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58708/0 (socket says 172.21.15.85:58708) 2020-11-20T13:55:33.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 -- 172.21.15.85:0/309760342 learned_addr learned my addr 172.21.15.85:0/309760342 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:33.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42892700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c05f300 0x7f0e3c090e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:33.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 -- 172.21.15.85:0/309760342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c05f300 msgr2=0x7f0e3c090e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c05f300 0x7f0e3c090e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.991 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 -- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 msgr2=0x7f0e3c0b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:33.992 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c0b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:33.992 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 -- 172.21.15.85:0/309760342 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c090960 con 0x7f0e3c099fc0 2020-11-20T13:55:33.992 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e43093700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c0b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:33.992 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e42091700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a3d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0e38002f30 tx=0x7f0e38015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:33.992 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.978+0000 7f0e337fe700 1 -- 172.21.15.85:0/309760342 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e38006bb0 con 0x7f0e3c099fc0 2020-11-20T13:55:33.993 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.979+0000 7f0e337fe700 1 -- 172.21.15.85:0/309760342 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38011c30 con 0x7f0e3c099fc0 2020-11-20T13:55:33.993 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.979+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c0b44e0 con 0x7f0e3c099fc0 2020-11-20T13:55:33.993 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.979+0000 7f0e337fe700 1 -- 172.21.15.85:0/309760342 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e3801e460 con 0x7f0e3c099fc0 2020-11-20T13:55:33.993 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.979+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c09ab00 con 0x7f0e3c099fc0 2020-11-20T13:55:33.993 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.979+0000 7f0e337fe700 1 -- 172.21.15.85:0/309760342 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f0e38025030 con 0x7f0e3c099fc0 2020-11-20T13:55:33.994 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.979+0000 7f0e337fe700 1 -- 172.21.15.85:0/309760342 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f0e3801d970 con 0x7f0e3c099fc0 2020-11-20T13:55:33.994 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.980+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c090af0 con 0x7f0e3c099fc0 2020-11-20T13:55:33.994 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:33.982+0000 7f0e337fe700 1 -- 172.21.15.85:0/309760342 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f0e38023070 con 0x7f0e3c099fc0 2020-11-20T13:55:34.070 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.069+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8e801f28-c346-40d6-a3f8-5e4d072ed72d", "id": 5} v 0) v1 -- 0x7f0e3c057050 con 0x7f0e3c099fc0 2020-11-20T13:55:34.073 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.072+0000 7f0e337fe700 1 -- 172.21.15.85:0/309760342 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8e801f28-c346-40d6-a3f8-5e4d072ed72d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0e3800b200 con 0x7f0e3c099fc0 2020-11-20T13:55:34.073 INFO:teuthology.orchestra.run.smithi085.stdout:5 2020-11-20T13:55:34.074 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 msgr2=0x7f0e3c09a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.075 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a3d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0e38002f30 tx=0x7f0e38015000).stop 2020-11-20T13:55:34.075 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 shutdown_connections 2020-11-20T13:55:34.075 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0e3c0b59a0 0x7f0e3c0b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.075 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e3c05f300 0x7f0e3c090e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.075 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 --2- 172.21.15.85:0/309760342 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0e3c099fc0 0x7f0e3c09a3d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.076 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 >> 172.21.15.85:0/309760342 conn(0x7f0e3c0b9490 msgr2=0x7f0e3c0b88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:34.076 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 shutdown_connections 2020-11-20T13:55:34.076 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.073+0000 7f0e442f5700 1 -- 172.21.15.85:0/309760342 wait complete. 2020-11-20T13:55:34.084 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 94604f91-e895-43c8-8032-36a36e1191c3 6 2020-11-20T13:55:34.199 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.195+0000 7fa86d2c2700 1 Processor -- start 2020-11-20T13:55:34.199 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.195+0000 7fa86d2c2700 1 -- start start 2020-11-20T13:55:34.199 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.195+0000 7fa86d2c2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8680b59a0 0x7fa868098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.199 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.195+0000 7fa86d2c2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa868099fc0 0x7fa86809a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.200 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.195+0000 7fa86d2c2700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8680995e0 con 0x7fa86805f300 2020-11-20T13:55:34.200 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.195+0000 7fa86d2c2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8680993c0 con 0x7fa868099fc0 2020-11-20T13:55:34.200 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.195+0000 7fa86d2c2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa868099880 con 0x7fa8680b59a0 2020-11-20T13:55:34.200 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa85effd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8680b59a0 0x7fa868098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.200 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa8677fe700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7fa86805f300 0x7fa86809b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:56702/0 (socket says 172.21.15.85:56702) 2020-11-20T13:55:34.201 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa8677fe700 1 -- 172.21.15.85:0/2782544671 learned_addr learned my addr 172.21.15.85:0/2782544671 (peer_addr_for_me v1:172.21.15.85:0/0) 2020-11-20T13:55:34.201 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa867fff700 1 --2- 172.21.15.85:0/2782544671 >> v2:172.21.15.117:3300/0 conn(0x7fa868099fc0 0x7fa86809a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.201 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa866ffd700 1 -- 172.21.15.85:0/2782544671 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378687150 0 0) 0x7fa8680995e0 con 0x7fa86805f300 2020-11-20T13:55:34.201 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa866ffd700 1 -- 172.21.15.85:0/2782544671 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa854003230 con 0x7fa86805f300 2020-11-20T13:55:34.202 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa85effd700 1 -- 172.21.15.85:0/2782544671 >> v2:172.21.15.117:3300/0 conn(0x7fa868099fc0 msgr2=0x7fa86809a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:34.202 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa85effd700 1 --2- 172.21.15.85:0/2782544671 >> v2:172.21.15.117:3300/0 conn(0x7fa868099fc0 0x7fa86809a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.202 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa85effd700 1 -- 172.21.15.85:0/2782544671 >> v1:172.21.15.85:6789/0 conn(0x7fa86805f300 legacy=0x7fa86809b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.202 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa85effd700 1 -- 172.21.15.85:0/2782544671 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa868099b20 con 0x7fa8680b59a0 2020-11-20T13:55:34.203 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa85effd700 1 --2- 172.21.15.85:0/2782544671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8680b59a0 0x7fa868098ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fa858002f30 tx=0x7fa858006dc0).ready entity=mon.2 client_cookie=a4ec7f83b19fdcdf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:34.203 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa866ffd700 1 -- 172.21.15.85:0/2782544671 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa8580096a0 con 0x7fa8680b59a0 2020-11-20T13:55:34.203 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa866ffd700 1 -- 172.21.15.85:0/2782544671 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858009800 con 0x7fa8680b59a0 2020-11-20T13:55:34.204 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa866ffd700 1 -- 172.21.15.85:0/2782544671 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa858009b60 con 0x7fa8680b59a0 2020-11-20T13:55:34.204 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa86d2c2700 1 -- 172.21.15.85:0/2782544671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8680b59a0 msgr2=0x7fa868098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.204 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.196+0000 7fa86d2c2700 1 --2- 172.21.15.85:0/2782544671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8680b59a0 0x7fa868098ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fa858002f30 tx=0x7fa858006dc0).stop 2020-11-20T13:55:34.204 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.197+0000 7fa86d2c2700 1 -- 172.21.15.85:0/2782544671 shutdown_connections 2020-11-20T13:55:34.205 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.197+0000 7fa86d2c2700 1 --2- 172.21.15.85:0/2782544671 >> v2:172.21.15.117:3300/0 conn(0x7fa868099fc0 0x7fa86809a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.205 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.197+0000 7fa86d2c2700 1 --2- 172.21.15.85:0/2782544671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa8680b59a0 0x7fa868098ed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.205 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.197+0000 7fa86d2c2700 1 -- 172.21.15.85:0/2782544671 >> 172.21.15.85:0/2782544671 conn(0x7fa8680b9490 msgr2=0x7fa8680b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:34.205 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.197+0000 7fa86d2c2700 1 -- 172.21.15.85:0/2782544671 shutdown_connections 2020-11-20T13:55:34.206 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.197+0000 7fa86d2c2700 1 -- 172.21.15.85:0/2782544671 wait complete. 2020-11-20T13:55:34.206 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.198+0000 7fa86d2c2700 1 Processor -- start 2020-11-20T13:55:34.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.198+0000 7fa86d2c2700 1 -- start start 2020-11-20T13:55:34.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa86805f300 0x7fa8680b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.208 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa868099fc0 0x7fa8680901f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.209 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8680b59a0 0x7fa8680662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.209 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680b45d0 con 0x7fa868099fc0 2020-11-20T13:55:34.209 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680b4860 con 0x7fa8680b59a0 2020-11-20T13:55:34.209 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680b4b00 con 0x7fa86805f300 2020-11-20T13:55:34.209 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa8677fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa86805f300 0x7fa8680b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.210 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa868099fc0 0x7fa8680901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.210 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa8677fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa86805f300 0x7fa8680b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58718/0 (socket says 172.21.15.85:58718) 2020-11-20T13:55:34.210 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa868099fc0 0x7fa8680901f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35768/0 (socket says 172.21.15.85:35768) 2020-11-20T13:55:34.210 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa8677fe700 1 -- 172.21.15.85:0/829383445 learned_addr learned my addr 172.21.15.85:0/829383445 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:34.211 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa867fff700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8680b59a0 0x7fa8680662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.211 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 -- 172.21.15.85:0/829383445 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa86805f300 msgr2=0x7fa8680b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.211 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa86805f300 0x7fa8680b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.211 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 -- 172.21.15.85:0/829383445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8680b59a0 msgr2=0x7fa8680662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.211 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8680b59a0 0x7fa8680662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.212 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 -- 172.21.15.85:0/829383445 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8680906e0 con 0x7fa868099fc0 2020-11-20T13:55:34.212 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa85effd700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa868099fc0 0x7fa8680901f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa858002f30 tx=0x7fa858015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:34.212 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa864ff9700 1 -- 172.21.15.85:0/829383445 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa858011ad0 con 0x7fa868099fc0 2020-11-20T13:55:34.212 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa864ff9700 1 -- 172.21.15.85:0/829383445 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858011c30 con 0x7fa868099fc0 2020-11-20T13:55:34.213 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa868090870 con 0x7fa868099fc0 2020-11-20T13:55:34.213 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa864ff9700 1 -- 172.21.15.85:0/829383445 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa85801d450 con 0x7fa868099fc0 2020-11-20T13:55:34.213 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.199+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa868066990 con 0x7fa868099fc0 2020-11-20T13:55:34.213 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.200+0000 7fa864ff9700 1 -- 172.21.15.85:0/829383445 <== mon.0 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 2) v1 ==== 31561+0+0 (secure 0 0 0) 0x7fa858024030 con 0x7fa868099fc0 2020-11-20T13:55:34.214 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.200+0000 7fa864ff9700 1 -- 172.21.15.85:0/829383445 <== mon.0 v2:172.21.15.85:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7fa858039e60 con 0x7fa868099fc0 2020-11-20T13:55:34.214 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.200+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa868057050 con 0x7fa868099fc0 2020-11-20T13:55:34.214 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.202+0000 7fa864ff9700 1 -- 172.21.15.85:0/829383445 <== mon.0 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fa858022070 con 0x7fa868099fc0 2020-11-20T13:55:34.290 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.289+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "94604f91-e895-43c8-8032-36a36e1191c3", "id": 6} v 0) v1 -- 0x7fa86805ca00 con 0x7fa868099fc0 2020-11-20T13:55:34.295 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.294+0000 7fa864ff9700 1 -- 172.21.15.85:0/829383445 <== mon.0 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "94604f91-e895-43c8-8032-36a36e1191c3", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa858022020 con 0x7fa868099fc0 2020-11-20T13:55:34.295 INFO:teuthology.orchestra.run.smithi085.stdout:6 2020-11-20T13:55:34.296 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa868099fc0 msgr2=0x7fa8680901f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.296 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa868099fc0 0x7fa8680901f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa858002f30 tx=0x7fa858015000).stop 2020-11-20T13:55:34.296 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 shutdown_connections 2020-11-20T13:55:34.297 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa86805f300 0x7fa8680b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.297 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8680b59a0 0x7fa8680662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.297 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 --2- 172.21.15.85:0/829383445 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa868099fc0 0x7fa8680901f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.297 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 >> 172.21.15.85:0/829383445 conn(0x7fa8680b9490 msgr2=0x7fa86809abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:34.298 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 shutdown_connections 2020-11-20T13:55:34.298 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.295+0000 7fa86d2c2700 1 -- 172.21.15.85:0/829383445 wait complete. 2020-11-20T13:55:34.306 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 0b70a2e6-1111-426a-a7d9-5ccdd88b4038 7 2020-11-20T13:55:34.413 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.409+0000 7f0b4f3dd700 1 Processor -- start 2020-11-20T13:55:34.413 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.409+0000 7f0b4f3dd700 1 -- start start 2020-11-20T13:55:34.413 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4f3dd700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f0b4805f300 0x7f0b48098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.414 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4f3dd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b480b59a0 0x7f0b4809b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.414 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4f3dd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b4809be70 0x7f0b4809d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.414 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4f3dd700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480993c0 con 0x7f0b4805f300 2020-11-20T13:55:34.414 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4f3dd700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480995e0 con 0x7f0b4809be70 2020-11-20T13:55:34.414 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4f3dd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48099880 con 0x7f0b480b59a0 2020-11-20T13:55:34.415 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b480b59a0 0x7f0b4809b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.415 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d97a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f0b4805f300 0x7f0b48098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.415 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b480b59a0 0x7f0b4809b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58724/0 (socket says 172.21.15.85:58724) 2020-11-20T13:55:34.415 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d97a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f0b4805f300 0x7f0b48098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:35772/0 (socket says 172.21.15.85:35772) 2020-11-20T13:55:34.415 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 -- 172.21.15.85:0/3725020529 learned_addr learned my addr 172.21.15.85:0/3725020529 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:34.416 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4e17b700 1 --2- 172.21.15.85:0/3725020529 >> v2:172.21.15.117:3300/0 conn(0x7f0b4809be70 0x7f0b4809d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.416 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 -- 172.21.15.85:0/3725020529 >> v2:172.21.15.117:3300/0 conn(0x7f0b4809be70 msgr2=0x7f0b4809d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:34.416 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 --2- 172.21.15.85:0/3725020529 >> v2:172.21.15.117:3300/0 conn(0x7f0b4809be70 0x7f0b4809d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.416 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 -- 172.21.15.85:0/3725020529 >> v2:172.21.15.85:3300/0 conn(0x7f0b4805f300 msgr2=0x7f0b48098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:34.416 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 --2- 172.21.15.85:0/3725020529 >> v2:172.21.15.85:3300/0 conn(0x7f0b4805f300 0x7f0b48098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.417 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 -- 172.21.15.85:0/3725020529 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48099b20 con 0x7f0b480b59a0 2020-11-20T13:55:34.417 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4e17b700 1 --2- 172.21.15.85:0/3725020529 >> v2:172.21.15.117:3300/0 conn(0x7f0b4809be70 0x7f0b4809d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:34.417 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d97a700 1 --2- 172.21.15.85:0/3725020529 >> v2:172.21.15.85:3300/0 conn(0x7f0b4805f300 0x7f0b48098ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:34.417 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4d179700 1 --2- 172.21.15.85:0/3725020529 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b480b59a0 0x7f0b4809b980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0b38002f30 tx=0x7f0b38006e80).ready entity=mon.2 client_cookie=cb192ae8163e7fe1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:34.417 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.410+0000 7f0b4c978700 1 -- 172.21.15.85:0/3725020529 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b380097f0 con 0x7f0b480b59a0 2020-11-20T13:55:34.418 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4c978700 1 -- 172.21.15.85:0/3725020529 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38009950 con 0x7f0b480b59a0 2020-11-20T13:55:34.418 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4c978700 1 -- 172.21.15.85:0/3725020529 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b38009cb0 con 0x7f0b480b59a0 2020-11-20T13:55:34.418 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/3725020529 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b480b59a0 msgr2=0x7f0b4809b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.418 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/3725020529 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b480b59a0 0x7f0b4809b980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0b38002f30 tx=0x7f0b38006e80).stop 2020-11-20T13:55:34.418 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/3725020529 shutdown_connections 2020-11-20T13:55:34.419 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/3725020529 >> v2:172.21.15.117:3300/0 conn(0x7f0b4809be70 0x7f0b4809d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.419 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/3725020529 >> v2:172.21.15.85:3300/0 conn(0x7f0b4805f300 0x7f0b48098ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.419 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/3725020529 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b480b59a0 0x7f0b4809b980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.419 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/3725020529 >> 172.21.15.85:0/3725020529 conn(0x7f0b480b9490 msgr2=0x7f0b480b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:34.419 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/3725020529 shutdown_connections 2020-11-20T13:55:34.420 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.411+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/3725020529 wait complete. 2020-11-20T13:55:34.422 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.412+0000 7f0b4f3dd700 1 Processor -- start 2020-11-20T13:55:34.423 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.412+0000 7f0b4f3dd700 1 -- start start 2020-11-20T13:55:34.423 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4f3dd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4805f300 0x7f0b480b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.423 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4f3dd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b4809be70 0x7f0b48145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.424 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4f3dd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b480b59a0 0x7f0b480b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:34.424 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4f3dd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480b4620 con 0x7f0b480b59a0 2020-11-20T13:55:34.424 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4f3dd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480b48c0 con 0x7f0b4809be70 2020-11-20T13:55:34.424 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4f3dd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480b4b60 con 0x7f0b4805f300 2020-11-20T13:55:34.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4805f300 0x7f0b480b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4805f300 0x7f0b480b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:58728/0 (socket says 172.21.15.85:58728) 2020-11-20T13:55:34.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 -- 172.21.15.85:0/1426097257 learned_addr learned my addr 172.21.15.85:0/1426097257 (peer_addr_for_me v2:172.21.15.85:0/0) 2020-11-20T13:55:34.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4e17b700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b480b59a0 0x7f0b480b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.425 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d179700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b4809be70 0x7f0b48145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:34.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 -- 172.21.15.85:0/1426097257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b4809be70 msgr2=0x7f0b48145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b4809be70 0x7f0b48145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 -- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b480b59a0 msgr2=0x7f0b480b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.426 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b480b59a0 0x7f0b480b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.427 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 -- 172.21.15.85:0/1426097257 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48146330 con 0x7f0b4805f300 2020-11-20T13:55:34.427 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d179700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b4809be70 0x7f0b48145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:34.427 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4e17b700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b480b59a0 0x7f0b480b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:34.427 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.413+0000 7f0b4d97a700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4805f300 0x7f0b480b5000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f0b4400c000 tx=0x7f0b44006dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:34.427 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.414+0000 7f0b3e7fc700 1 -- 172.21.15.85:0/1426097257 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b4400cb90 con 0x7f0b4805f300 2020-11-20T13:55:34.428 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.414+0000 7f0b3e7fc700 1 -- 172.21.15.85:0/1426097257 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4400ccf0 con 0x7f0b4805f300 2020-11-20T13:55:34.428 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.414+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b48065730 con 0x7f0b4805f300 2020-11-20T13:55:34.428 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.414+0000 7f0b3e7fc700 1 -- 172.21.15.85:0/1426097257 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b440125e0 con 0x7f0b4805f300 2020-11-20T13:55:34.428 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.414+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b480b1b80 con 0x7f0b4805f300 2020-11-20T13:55:34.428 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.414+0000 7f0b3e7fc700 1 -- 172.21.15.85:0/1426097257 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 2) v1 ==== 31561+0+0 (secure 0 0 0) 0x7f0b44012740 con 0x7f0b4805f300 2020-11-20T13:55:34.429 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.414+0000 7f0b3e7fc700 1 -- 172.21.15.85:0/1426097257 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1935+0+0 (secure 0 0 0) 0x7f0b44039720 con 0x7f0b4805f300 2020-11-20T13:55:34.429 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.415+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b48146580 con 0x7f0b4805f300 2020-11-20T13:55:34.429 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.417+0000 7f0b3e7fc700 1 -- 172.21.15.85:0/1426097257 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f0b48146580 con 0x7f0b4805f300 2020-11-20T13:55:34.501 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.499+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0b70a2e6-1111-426a-a7d9-5ccdd88b4038", "id": 7} v 0) v1 -- 0x7f0b48146580 con 0x7f0b4805f300 2020-11-20T13:55:34.508 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.507+0000 7f0b3e7fc700 1 -- 172.21.15.85:0/1426097257 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0b70a2e6-1111-426a-a7d9-5ccdd88b4038", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0b48146580 con 0x7f0b4805f300 2020-11-20T13:55:34.509 INFO:teuthology.orchestra.run.smithi085.stdout:7 2020-11-20T13:55:34.510 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.508+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4805f300 msgr2=0x7f0b480b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:34.510 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.508+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4805f300 0x7f0b480b5000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f0b4400c000 tx=0x7f0b44006dc0).stop 2020-11-20T13:55:34.510 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.508+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 shutdown_connections 2020-11-20T13:55:34.510 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.508+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0b4805f300 0x7f0b480b5000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.511 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.508+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0b480b59a0 0x7f0b480b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.511 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.508+0000 7f0b4f3dd700 1 --2- 172.21.15.85:0/1426097257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b4809be70 0x7f0b48145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:34.511 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.508+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 >> 172.21.15.85:0/1426097257 conn(0x7f0b480b9490 msgr2=0x7f0b48062430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:34.511 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.509+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 shutdown_connections 2020-11-20T13:55:34.512 INFO:teuthology.orchestra.run.smithi085.stderr:2020-11-20T13:55:34.509+0000 7f0b4f3dd700 1 -- 172.21.15.85:0/1426097257 wait complete. 2020-11-20T13:55:34.519 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-20T13:55:34.520 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-20T13:55:34.522 INFO:tasks.ceph.osd.0:Started 2020-11-20T13:55:34.523 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-20T13:55:34.523 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-20T13:55:34.525 INFO:tasks.ceph.osd.1:Started 2020-11-20T13:55:34.527 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-20T13:55:34.528 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-20T13:55:34.529 INFO:tasks.ceph.osd.2:Started 2020-11-20T13:55:34.530 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-20T13:55:34.530 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-20T13:55:34.532 INFO:tasks.ceph.osd.3:Started 2020-11-20T13:55:34.533 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-20T13:55:34.533 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-20T13:55:34.535 INFO:tasks.ceph.osd.4:Started 2020-11-20T13:55:34.535 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-20T13:55:34.536 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-20T13:55:34.538 INFO:tasks.ceph.osd.5:Started 2020-11-20T13:55:34.538 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-20T13:55:34.538 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-20T13:55:34.540 INFO:tasks.ceph.osd.6:Started 2020-11-20T13:55:34.541 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-20T13:55:34.541 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-20T13:55:34.543 INFO:tasks.ceph.osd.7:Started 2020-11-20T13:55:34.543 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-20T13:55:35.124 INFO:tasks.ceph.osd.0.smithi117.stderr:2020-11-20T13:55:35.123+0000 7f25fd713ec0 -1 Falling back to public interface 2020-11-20T13:55:35.132 INFO:tasks.ceph.osd.2.smithi117.stderr:2020-11-20T13:55:35.131+0000 7ff4cd4bfec0 -1 Falling back to public interface 2020-11-20T13:55:35.136 INFO:tasks.ceph.osd.1.smithi117.stderr:2020-11-20T13:55:35.135+0000 7f8fb9f0aec0 -1 Falling back to public interface 2020-11-20T13:55:35.143 INFO:tasks.ceph.osd.3.smithi117.stderr:2020-11-20T13:55:35.143+0000 7f0915383ec0 -1 Falling back to public interface 2020-11-20T13:55:35.145 INFO:tasks.ceph.osd.4.smithi085.stderr:2020-11-20T13:55:35.144+0000 7f25cd231ec0 -1 Falling back to public interface 2020-11-20T13:55:35.149 INFO:tasks.ceph.osd.7.smithi085.stderr:2020-11-20T13:55:35.148+0000 7f164fb85ec0 -1 Falling back to public interface 2020-11-20T13:55:35.151 INFO:tasks.ceph.osd.6.smithi085.stderr:2020-11-20T13:55:35.150+0000 7f1821c06ec0 -1 Falling back to public interface 2020-11-20T13:55:35.157 INFO:tasks.ceph.osd.5.smithi085.stderr:2020-11-20T13:55:35.156+0000 7f145b93dec0 -1 Falling back to public interface 2020-11-20T13:55:35.350 INFO:teuthology.orchestra.run.smithi117:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T13:55:35.451 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.448+0000 7fe27829a700 1 Processor -- start 2020-11-20T13:55:35.451 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.448+0000 7fe27829a700 1 -- start start 2020-11-20T13:55:35.451 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe27829a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe270080410 0x7fe2700624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:35.452 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe27829a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2700807f0 0x7fe27009b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:35.452 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe27829a700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27009ab50 con 0x7fe27009b9b0 2020-11-20T13:55:35.452 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe27829a700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27009ad80 con 0x7fe270080410 2020-11-20T13:55:35.452 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe27829a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27009b020 con 0x7fe2700807f0 2020-11-20T13:55:35.452 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2700807f0 0x7fe27009b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:35.453 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2700807f0 0x7fe27009b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:39652/0 (socket says 172.21.15.117:39652) 2020-11-20T13:55:35.453 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 -- 172.21.15.117:0/1261725683 learned_addr learned my addr 172.21.15.117:0/1261725683 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:35.453 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe277038700 1 --1- 172.21.15.117:0/1261725683 >> v1:172.21.15.85:6789/0 conn(0x7fe27009b9b0 0x7fe270099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:42654/0 (socket says 172.21.15.117:42654) 2020-11-20T13:55:35.453 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276837700 1 --2- 172.21.15.117:0/1261725683 >> v2:172.21.15.85:3300/0 conn(0x7fe270080410 0x7fe2700624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:35.453 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 -- 172.21.15.117:0/1261725683 >> v2:172.21.15.85:3300/0 conn(0x7fe270080410 msgr2=0x7fe2700624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:35.454 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 --2- 172.21.15.117:0/1261725683 >> v2:172.21.15.85:3300/0 conn(0x7fe270080410 0x7fe2700624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.454 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 -- 172.21.15.117:0/1261725683 >> v1:172.21.15.85:6789/0 conn(0x7fe27009b9b0 legacy=0x7fe270099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:35.454 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 -- 172.21.15.117:0/1261725683 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2700629b0 con 0x7fe2700807f0 2020-11-20T13:55:35.454 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.449+0000 7fe276036700 1 --2- 172.21.15.117:0/1261725683 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2700807f0 0x7fe27009b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fe260002f30 tx=0x7fe260006e80).ready entity=mon.2 client_cookie=3154298cd397dfa8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:35.454 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe275835700 1 -- 172.21.15.117:0/1261725683 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2600097f0 con 0x7fe2700807f0 2020-11-20T13:55:35.455 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe275835700 1 -- 172.21.15.117:0/1261725683 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe260009950 con 0x7fe2700807f0 2020-11-20T13:55:35.455 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe275835700 1 -- 172.21.15.117:0/1261725683 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe260009cb0 con 0x7fe2700807f0 2020-11-20T13:55:35.455 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 -- 172.21.15.117:0/1261725683 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2700807f0 msgr2=0x7fe27009b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:35.455 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 --2- 172.21.15.117:0/1261725683 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2700807f0 0x7fe27009b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fe260002f30 tx=0x7fe260006e80).stop 2020-11-20T13:55:35.456 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 -- 172.21.15.117:0/1261725683 shutdown_connections 2020-11-20T13:55:35.456 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 --2- 172.21.15.117:0/1261725683 >> v2:172.21.15.85:3300/0 conn(0x7fe270080410 0x7fe2700624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.456 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 --2- 172.21.15.117:0/1261725683 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2700807f0 0x7fe27009b4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.456 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 -- 172.21.15.117:0/1261725683 >> 172.21.15.117:0/1261725683 conn(0x7fe27005c4f0 msgr2=0x7fe27005f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:35.456 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 -- 172.21.15.117:0/1261725683 shutdown_connections 2020-11-20T13:55:35.457 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.450+0000 7fe27829a700 1 -- 172.21.15.117:0/1261725683 wait complete. 2020-11-20T13:55:35.460 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.451+0000 7fe27829a700 1 Processor -- start 2020-11-20T13:55:35.460 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.451+0000 7fe27829a700 1 -- start start 2020-11-20T13:55:35.460 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe27829a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe270080410 0x7fe27015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:35.460 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe27829a700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2700807f0 0x7fe2701573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:35.460 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe27829a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe27009b9b0 0x7fe270158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:35.461 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe27829a700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2701578c0 con 0x7fe2700807f0 2020-11-20T13:55:35.461 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe27829a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe270157b50 con 0x7fe270080410 2020-11-20T13:55:35.461 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe27829a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe270157df0 con 0x7fe27009b9b0 2020-11-20T13:55:35.461 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe270080410 0x7fe27015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:35.462 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe270080410 0x7fe27015dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39592/0 (socket says 172.21.15.117:39592) 2020-11-20T13:55:35.462 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 -- 172.21.15.117:0/3535807888 learned_addr learned my addr 172.21.15.117:0/3535807888 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:35.462 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276036700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2700807f0 0x7fe2701573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:35.462 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe277038700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe27009b9b0 0x7fe270158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:35.462 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 -- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe27009b9b0 msgr2=0x7fe270158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:35.463 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe27009b9b0 0x7fe270158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.463 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 -- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2700807f0 msgr2=0x7fe2701573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:35.463 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2700807f0 0x7fe2701573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.463 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276837700 1 -- 172.21.15.117:0/3535807888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe270158090 con 0x7fe270080410 2020-11-20T13:55:35.463 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.452+0000 7fe276036700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2700807f0 0x7fe2701573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:35.464 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.453+0000 7fe276837700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe270080410 0x7fe27015dd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fe26c00c000 tx=0x7fe26c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:35.464 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.453+0000 7fe2677fe700 1 -- 172.21.15.117:0/3535807888 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe26c00ca10 con 0x7fe270080410 2020-11-20T13:55:35.464 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.453+0000 7fe2677fe700 1 -- 172.21.15.117:0/3535807888 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c00cb70 con 0x7fe270080410 2020-11-20T13:55:35.464 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.453+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe270158a20 con 0x7fe270080410 2020-11-20T13:55:35.464 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.453+0000 7fe2677fe700 1 -- 172.21.15.117:0/3535807888 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe26c0123f0 con 0x7fe270080410 2020-11-20T13:55:35.465 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.453+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2700ae610 con 0x7fe270080410 2020-11-20T13:55:35.465 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.454+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe270057050 con 0x7fe270080410 2020-11-20T13:55:35.465 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.454+0000 7fe2677fe700 1 -- 172.21.15.117:0/3535807888 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 3) v1 ==== 63226+0+0 (secure 0 0 0) 0x7fe26c012550 con 0x7fe270080410 2020-11-20T13:55:35.465 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.455+0000 7fe2677fe700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe258051130 0x7fe2580535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:35.465 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.455+0000 7fe2677fe700 1 -- 172.21.15.117:0/3535807888 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7fe26c068180 con 0x7fe270080410 2020-11-20T13:55:35.466 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.455+0000 7fe276036700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe258051130 0x7fe2580535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:35.466 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.455+0000 7fe276036700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe258051130 0x7fe2580535f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe26001c000 tx=0x7fe260015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:35.466 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.458+0000 7fe2677fe700 1 -- 172.21.15.117:0/3535807888 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe26c041520 con 0x7fe270080410 2020-11-20T13:55:35.613 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.613+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe27005d700 con 0x7fe270080410 2020-11-20T13:55:35.614 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.614+0000 7fe2677fe700 1 -- 172.21.15.117:0/3535807888 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+8275 (secure 0 0 0) 0x7fe26c044b40 con 0x7fe270080410 2020-11-20T13:55:35.614 INFO:teuthology.misc.health.smithi117.stdout: 2020-11-20T13:55:35.615 INFO:teuthology.misc.health.smithi117.stdout:{"epoch":10,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:34.501303+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:35.616 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe258051130 msgr2=0x7fe2580535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:35.616 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe258051130 0x7fe2580535f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe26001c000 tx=0x7fe260015000).stop 2020-11-20T13:55:35.616 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe270080410 msgr2=0x7fe27015dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:35.616 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe270080410 0x7fe27015dd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fe26c00c000 tx=0x7fe26c006dc0).stop 2020-11-20T13:55:35.617 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 shutdown_connections 2020-11-20T13:55:35.617 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe258051130 0x7fe2580535f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.617 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe27009b9b0 0x7fe270158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.617 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe270080410 0x7fe27015dd60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.617 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 --2- 172.21.15.117:0/3535807888 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2700807f0 0x7fe2701573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:35.618 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 >> 172.21.15.117:0/3535807888 conn(0x7fe27005c4f0 msgr2=0x7fe270099d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:35.618 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 shutdown_connections 2020-11-20T13:55:35.618 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:35.615+0000 7fe27829a700 1 -- 172.21.15.117:0/3535807888 wait complete. 2020-11-20T13:55:35.625 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-20T13:55:36.191 INFO:tasks.ceph.osd.0.smithi117.stderr:2020-11-20T13:55:36.191+0000 7f25fd713ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-11-20T13:55:36.212 INFO:tasks.ceph.osd.2.smithi117.stderr:2020-11-20T13:55:36.212+0000 7ff4cd4bfec0 -1 osd.2 0 log_to_monitors {default=true} 2020-11-20T13:55:36.224 INFO:tasks.ceph.osd.3.smithi117.stderr:2020-11-20T13:55:36.224+0000 7f0915383ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-11-20T13:55:36.224 INFO:tasks.ceph.osd.1.smithi117.stderr:2020-11-20T13:55:36.224+0000 7f8fb9f0aec0 -1 osd.1 0 log_to_monitors {default=true} 2020-11-20T13:55:36.225 INFO:tasks.ceph.osd.4.smithi085.stderr:2020-11-20T13:55:36.225+0000 7f25cd231ec0 -1 osd.4 0 log_to_monitors {default=true} 2020-11-20T13:55:36.237 INFO:tasks.ceph.osd.7.smithi085.stderr:2020-11-20T13:55:36.236+0000 7f164fb85ec0 -1 osd.7 0 log_to_monitors {default=true} 2020-11-20T13:55:36.239 INFO:tasks.ceph.osd.6.smithi085.stderr:2020-11-20T13:55:36.239+0000 7f1821c06ec0 -1 osd.6 0 log_to_monitors {default=true} 2020-11-20T13:55:36.249 INFO:tasks.ceph.osd.5.smithi085.stderr:2020-11-20T13:55:36.243+0000 7f145b93dec0 -1 osd.5 0 log_to_monitors {default=true} 2020-11-20T13:55:38.147 INFO:tasks.ceph.osd.2.smithi117.stderr:2020-11-20T13:55:38.147+0000 7ff4b5a42700 -1 osd.2 0 waiting for initial osdmap 2020-11-20T13:55:38.148 INFO:tasks.ceph.osd.5.smithi085.stderr:2020-11-20T13:55:38.146+0000 7f1443ec0700 -1 osd.5 0 waiting for initial osdmap 2020-11-20T13:55:38.148 INFO:tasks.ceph.osd.3.smithi117.stderr:2020-11-20T13:55:38.148+0000 7f08fd906700 -1 osd.3 0 waiting for initial osdmap 2020-11-20T13:55:38.149 INFO:tasks.ceph.osd.6.smithi085.stderr:2020-11-20T13:55:38.148+0000 7f180a189700 -1 osd.6 0 waiting for initial osdmap 2020-11-20T13:55:38.149 INFO:tasks.ceph.osd.4.smithi085.stderr:2020-11-20T13:55:38.148+0000 7f25b57b4700 -1 osd.4 0 waiting for initial osdmap 2020-11-20T13:55:38.150 INFO:tasks.ceph.osd.1.smithi117.stderr:2020-11-20T13:55:38.149+0000 7f8fa248d700 -1 osd.1 0 waiting for initial osdmap 2020-11-20T13:55:38.151 INFO:tasks.ceph.osd.0.smithi117.stderr:2020-11-20T13:55:38.151+0000 7f25e5c96700 -1 osd.0 0 waiting for initial osdmap 2020-11-20T13:55:38.153 INFO:tasks.ceph.osd.7.smithi085.stderr:2020-11-20T13:55:38.150+0000 7f1638108700 -1 osd.7 0 waiting for initial osdmap 2020-11-20T13:55:42.432 INFO:teuthology.orchestra.run.smithi117:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T13:55:42.540 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.537+0000 7f05e410b700 1 Processor -- start 2020-11-20T13:55:42.540 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.537+0000 7f05e410b700 1 -- start start 2020-11-20T13:55:42.541 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e410b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05dc080410 0x7f05dc0624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.541 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e410b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc0807f0 0x7f05dc09b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.541 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e410b700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05dc09ab50 con 0x7f05dc09b9b0 2020-11-20T13:55:42.541 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e410b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc09ad80 con 0x7f05dc080410 2020-11-20T13:55:42.542 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e410b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc09b020 con 0x7f05dc0807f0 2020-11-20T13:55:42.542 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e26a8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05dc080410 0x7f05dc0624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.542 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e26a8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05dc080410 0x7f05dc0624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39816/0 (socket says 172.21.15.117:39816) 2020-11-20T13:55:42.542 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.538+0000 7f05e26a8700 1 -- 172.21.15.117:0/3876092975 learned_addr learned my addr 172.21.15.117:0/3876092975 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:42.542 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e1ea7700 1 --2- 172.21.15.117:0/3876092975 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc0807f0 0x7f05dc09b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.543 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e26a8700 1 -- 172.21.15.117:0/3876092975 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc0807f0 msgr2=0x7f05dc09b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:42.543 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e26a8700 1 --2- 172.21.15.117:0/3876092975 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc0807f0 0x7f05dc09b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.543 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e26a8700 1 -- 172.21.15.117:0/3876092975 >> v1:172.21.15.85:6789/0 conn(0x7f05dc09b9b0 legacy=0x7f05dc099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:42.543 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e26a8700 1 -- 172.21.15.117:0/3876092975 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc0629b0 con 0x7f05dc080410 2020-11-20T13:55:42.544 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e1ea7700 1 --2- 172.21.15.117:0/3876092975 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc0807f0 0x7f05dc09b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:42.544 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e26a8700 1 --2- 172.21.15.117:0/3876092975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0624c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f05dc061f00 tx=0x7f05d8006e80).ready entity=mon.1 client_cookie=8025e54cef4cbc9a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:42.544 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e16a6700 1 -- 172.21.15.117:0/3876092975 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d80097b0 con 0x7f05dc080410 2020-11-20T13:55:42.544 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e16a6700 1 -- 172.21.15.117:0/3876092975 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d8009910 con 0x7f05dc080410 2020-11-20T13:55:42.544 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e16a6700 1 -- 172.21.15.117:0/3876092975 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d8009ac0 con 0x7f05dc080410 2020-11-20T13:55:42.545 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 -- 172.21.15.117:0/3876092975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 msgr2=0x7f05dc0624c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.545 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 --2- 172.21.15.117:0/3876092975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0624c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f05dc061f00 tx=0x7f05d8006e80).stop 2020-11-20T13:55:42.545 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 -- 172.21.15.117:0/3876092975 shutdown_connections 2020-11-20T13:55:42.545 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 --2- 172.21.15.117:0/3876092975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0624c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.545 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 --2- 172.21.15.117:0/3876092975 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc0807f0 0x7f05dc09b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.546 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 -- 172.21.15.117:0/3876092975 >> 172.21.15.117:0/3876092975 conn(0x7f05dc05c4f0 msgr2=0x7f05dc05f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:42.546 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 -- 172.21.15.117:0/3876092975 shutdown_connections 2020-11-20T13:55:42.546 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.539+0000 7f05e410b700 1 -- 172.21.15.117:0/3876092975 wait complete. 2020-11-20T13:55:42.548 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 Processor -- start 2020-11-20T13:55:42.549 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 -- start start 2020-11-20T13:55:42.549 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.549 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05dc0807f0 0x7f05dc145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.549 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc09b9b0 0x7f05dc0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.550 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc135c60 con 0x7f05dc0807f0 2020-11-20T13:55:42.550 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0b22f0 con 0x7f05dc080410 2020-11-20T13:55:42.550 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e410b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0b2590 con 0x7f05dc09b9b0 2020-11-20T13:55:42.550 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e26a8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.550 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e26a8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39822/0 (socket says 172.21.15.117:39822) 2020-11-20T13:55:42.551 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.541+0000 7f05e26a8700 1 -- 172.21.15.117:0/451741628 learned_addr learned my addr 172.21.15.117:0/451741628 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:42.551 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e1ea7700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05dc0807f0 0x7f05dc145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.551 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e2ea9700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc09b9b0 0x7f05dc0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.551 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e26a8700 1 -- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc09b9b0 msgr2=0x7f05dc0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.552 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e26a8700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc09b9b0 0x7f05dc0b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.552 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e26a8700 1 -- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05dc0807f0 msgr2=0x7f05dc145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.552 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e26a8700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05dc0807f0 0x7f05dc145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.552 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e26a8700 1 -- 172.21.15.117:0/451741628 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc0b2830 con 0x7f05dc080410 2020-11-20T13:55:42.552 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e26a8700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0650f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f05dc061f00 tx=0x7f05d8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:42.553 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05d2ffd700 1 -- 172.21.15.117:0/451741628 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d8011db0 con 0x7f05dc080410 2020-11-20T13:55:42.553 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05d2ffd700 1 -- 172.21.15.117:0/451741628 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d801e400 con 0x7f05dc080410 2020-11-20T13:55:42.553 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05d2ffd700 1 -- 172.21.15.117:0/451741628 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d801e760 con 0x7f05dc080410 2020-11-20T13:55:42.553 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc146080 con 0x7f05dc080410 2020-11-20T13:55:42.553 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.542+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc0ae680 con 0x7f05dc080410 2020-11-20T13:55:42.554 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.543+0000 7f05d2ffd700 1 -- 172.21.15.117:0/451741628 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f05d802e070 con 0x7f05dc080410 2020-11-20T13:55:42.554 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.543+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05dc057050 con 0x7f05dc080410 2020-11-20T13:55:42.554 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.543+0000 7f05d2ffd700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05c4051200 0x7f05c40536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.554 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.544+0000 7f05d2ffd700 1 -- 172.21.15.117:0/451741628 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f05d806bf00 con 0x7f05dc080410 2020-11-20T13:55:42.555 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.544+0000 7f05e1ea7700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05c4051200 0x7f05c40536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.555 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.544+0000 7f05e1ea7700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05c4051200 0x7f05c40536c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f05cc00b000 tx=0x7f05cc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:42.555 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.547+0000 7f05d2ffd700 1 -- 172.21.15.117:0/451741628 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f05d8044bb0 con 0x7f05dc080410 2020-11-20T13:55:42.697 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.697+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05dc0b29c0 con 0x7f05dc080410 2020-11-20T13:55:42.698 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.697+0000 7f05d2ffd700 1 -- 172.21.15.117:0/451741628 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11853 (secure 0 0 0) 0x7f05d8022020 con 0x7f05dc080410 2020-11-20T13:55:42.698 INFO:teuthology.misc.health.smithi117.stdout: 2020-11-20T13:55:42.698 INFO:teuthology.misc.health.smithi117.stdout:{"epoch":15,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:41.162172+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:42.699 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05c4051200 msgr2=0x7f05c40536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.700 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05c4051200 0x7f05c40536c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f05cc00b000 tx=0x7f05cc008040).stop 2020-11-20T13:55:42.700 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 msgr2=0x7f05dc0650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.700 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0650f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f05dc061f00 tx=0x7f05d8016000).stop 2020-11-20T13:55:42.700 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 shutdown_connections 2020-11-20T13:55:42.701 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05c4051200 0x7f05c40536c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.701 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f05dc09b9b0 0x7f05dc0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.701 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05dc080410 0x7f05dc0650f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.702 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 --2- 172.21.15.117:0/451741628 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f05dc0807f0 0x7f05dc145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.702 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 >> 172.21.15.117:0/451741628 conn(0x7f05dc05c4f0 msgr2=0x7f05dc099d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:42.702 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 shutdown_connections 2020-11-20T13:55:42.702 INFO:teuthology.misc.health.smithi117.stderr:2020-11-20T13:55:42.699+0000 7f05e410b700 1 -- 172.21.15.117:0/451741628 wait complete. 2020-11-20T13:55:42.708 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-20T13:55:42.708 INFO:tasks.ceph:Creating RBD pool 2020-11-20T13:55:42.709 INFO:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-20T13:55:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.814+0000 7f2c754d2700 1 Processor -- start 2020-11-20T13:55:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.814+0000 7f2c754d2700 1 -- start start 2020-11-20T13:55:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.814+0000 7f2c754d2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c7005f2d0 0x7f2c700b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c754d2700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c700af250 con 0x7f2c700b4300 2020-11-20T13:55:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c754d2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c70098ab0 con 0x7f2c700ae930 2020-11-20T13:55:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c754d2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c700991f0 con 0x7f2c7005f2d0 2020-11-20T13:55:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c7005f2d0 0x7f2c700b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c66ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2c700ae930 0x7f2c7009b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:58990/0 (socket says 172.21.15.117:58990) 2020-11-20T13:55:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c66ffd700 1 -- 172.21.15.117:0/3795619302 learned_addr learned my addr 172.21.15.117:0/3795619302 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c7005f2d0 0x7f2c700b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39830/0 (socket says 172.21.15.117:39830) 2020-11-20T13:55:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6f7fe700 1 -- 172.21.15.117:0/3795619302 >> v1:172.21.15.117:6789/0 conn(0x7f2c700ae930 legacy=0x7f2c7009b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6f7fe700 1 -- 172.21.15.117:0/3795619302 >> v1:172.21.15.85:6789/0 conn(0x7f2c700b4300 legacy=0x7f2c7015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6effd700 1 -- 172.21.15.117:0/3795619302 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082034040 0 0) 0x7f2c70098ab0 con 0x7f2c700ae930 2020-11-20T13:55:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6f7fe700 1 -- 172.21.15.117:0/3795619302 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c70098d50 con 0x7f2c7005f2d0 2020-11-20T13:55:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6f7fe700 1 --2- 172.21.15.117:0/3795619302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b3e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2c58002f30 tx=0x7f2c58006dc0).ready entity=mon.1 client_cookie=c62384c5733b9f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6effd700 1 -- 172.21.15.117:0/3795619302 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c580097f0 con 0x7f2c7005f2d0 2020-11-20T13:55:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6effd700 1 -- 172.21.15.117:0/3795619302 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58009950 con 0x7f2c7005f2d0 2020-11-20T13:55:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c6effd700 1 -- 172.21.15.117:0/3795619302 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c58009cb0 con 0x7f2c7005f2d0 2020-11-20T13:55:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c754d2700 1 -- 172.21.15.117:0/3795619302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 msgr2=0x7f2c700b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.815+0000 7f2c754d2700 1 --2- 172.21.15.117:0/3795619302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b3e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2c58002f30 tx=0x7f2c58006dc0).stop 2020-11-20T13:55:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.816+0000 7f2c754d2700 1 -- 172.21.15.117:0/3795619302 shutdown_connections 2020-11-20T13:55:42.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.816+0000 7f2c754d2700 1 --2- 172.21.15.117:0/3795619302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b3e10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.816+0000 7f2c754d2700 1 -- 172.21.15.117:0/3795619302 >> 172.21.15.117:0/3795619302 conn(0x7f2c700b9470 msgr2=0x7f2c700b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:42.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.816+0000 7f2c754d2700 1 -- 172.21.15.117:0/3795619302 shutdown_connections 2020-11-20T13:55:42.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.816+0000 7f2c754d2700 1 -- 172.21.15.117:0/3795619302 wait complete. 2020-11-20T13:55:42.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.817+0000 7f2c754d2700 1 Processor -- start 2020-11-20T13:55:42.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.817+0000 7f2c754d2700 1 -- start start 2020-11-20T13:55:42.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c700ae930 0x7f2c70062460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c700b4300 0x7f2c70066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70062950 con 0x7f2c700ae930 2020-11-20T13:55:42.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70062be0 con 0x7f2c7005f2d0 2020-11-20T13:55:42.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70062e80 con 0x7f2c700b4300 2020-11-20T13:55:42.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39836/0 (socket says 172.21.15.117:39836) 2020-11-20T13:55:42.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 -- 172.21.15.117:0/608419258 learned_addr learned my addr 172.21.15.117:0/608419258 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:42.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6ffff700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c700b4300 0x7f2c70066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 -- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c700b4300 msgr2=0x7f2c70066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c66ffd700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c700ae930 0x7f2c70062460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c700b4300 0x7f2c70066640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 -- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c700ae930 msgr2=0x7f2c70062460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:42.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c700ae930 0x7f2c70062460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:42.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 -- 172.21.15.117:0/608419258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c70066b30 con 0x7f2c7005f2d0 2020-11-20T13:55:42.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6f7fe700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b2b00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2c58002f30 tx=0x7f2c58015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:42.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6cff9700 1 -- 172.21.15.117:0/608419258 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c5801d450 con 0x7f2c7005f2d0 2020-11-20T13:55:42.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6cff9700 1 -- 172.21.15.117:0/608419258 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5801d5b0 con 0x7f2c7005f2d0 2020-11-20T13:55:42.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c70063850 con 0x7f2c7005f2d0 2020-11-20T13:55:42.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c6cff9700 1 -- 172.21.15.117:0/608419258 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c5801d910 con 0x7f2c7005f2d0 2020-11-20T13:55:42.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.818+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c70157700 con 0x7f2c7005f2d0 2020-11-20T13:55:42.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.819+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c70063b50 con 0x7f2c7005f2d0 2020-11-20T13:55:42.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.820+0000 7f2c6cff9700 1 -- 172.21.15.117:0/608419258 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2c5800f070 con 0x7f2c7005f2d0 2020-11-20T13:55:42.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.820+0000 7f2c6cff9700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2c54051320 0x7f2c540537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:42.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.820+0000 7f2c6cff9700 1 -- 172.21.15.117:0/608419258 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f2c58069db0 con 0x7f2c7005f2d0 2020-11-20T13:55:42.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.820+0000 7f2c66ffd700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2c54051320 0x7f2c540537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:42.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.820+0000 7f2c66ffd700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2c54051320 0x7f2c540537e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2c60008000 tx=0x7f2c60009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:42.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.823+0000 7f2c6cff9700 1 -- 172.21.15.117:0/608419258 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2c580425a0 con 0x7f2c7005f2d0 2020-11-20T13:55:42.972 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:42.972+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f2c70063040 con 0x7f2c7005f2d0 2020-11-20T13:55:43.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.185+0000 7f2c6cff9700 1 -- 172.21.15.117:0/608419258 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f2c58022030 con 0x7f2c7005f2d0 2020-11-20T13:55:43.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.187+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2c54051320 msgr2=0x7f2c540537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:43.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.187+0000 7f2c754d2700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2c54051320 0x7f2c540537e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2c60008000 tx=0x7f2c60009040).stop 2020-11-20T13:55:43.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.187+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 msgr2=0x7f2c700b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:43.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.187+0000 7f2c754d2700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b2b00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2c58002f30 tx=0x7f2c58015000).stop 2020-11-20T13:55:43.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 shutdown_connections 2020-11-20T13:55:43.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2c54051320 0x7f2c540537e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c700b4300 0x7f2c70066640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7005f2d0 0x7f2c700b2b00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 --2- 172.21.15.117:0/608419258 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c700ae930 0x7f2c70062460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 >> 172.21.15.117:0/608419258 conn(0x7f2c700b9470 msgr2=0x7f2c700b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 shutdown_connections 2020-11-20T13:55:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.188+0000 7f2c754d2700 1 -- 172.21.15.117:0/608419258 wait complete. 2020-11-20T13:55:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'rbd' created 2020-11-20T13:55:43.200 INFO:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-20T13:55:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.307+0000 7fa390df4700 1 Processor -- start 2020-11-20T13:55:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.307+0000 7fa390df4700 1 -- start start 2020-11-20T13:55:43.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa390df4700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa38c060580 0x7fa38c09a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa390df4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c0af850 0x7fa38c0b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa390df4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38c080b70 con 0x7fa38c025950 2020-11-20T13:55:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa390df4700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38c0610d0 con 0x7fa38c060580 2020-11-20T13:55:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa390df4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c060960 con 0x7fa38c0af850 2020-11-20T13:55:43.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa38c060580 0x7fa38c09a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:43.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa383fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa38c025950 0x7fa38c09cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59004/0 (socket says 172.21.15.117:59004) 2020-11-20T13:55:43.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa383fff700 1 -- 172.21.15.117:0/3783811867 learned_addr learned my addr 172.21.15.117:0/3783811867 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:43.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa392056700 1 --2- 172.21.15.117:0/3783811867 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c0af850 0x7fa38c0b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:43.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38bfff700 1 -- 172.21.15.117:0/3783811867 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c0af850 msgr2=0x7fa38c0b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:43.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38bfff700 1 --2- 172.21.15.117:0/3783811867 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c0af850 0x7fa38c0b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38bfff700 1 -- 172.21.15.117:0/3783811867 >> v1:172.21.15.117:6789/0 conn(0x7fa38c025950 legacy=0x7fa38c09cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:43.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38bfff700 1 -- 172.21.15.117:0/3783811867 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c060ea0 con 0x7fa38c060580 2020-11-20T13:55:43.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38b7fe700 1 -- 172.21.15.117:0/3783811867 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496270883 0 0) 0x7fa38c080b70 con 0x7fa38c025950 2020-11-20T13:55:43.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38bfff700 1 --2- 172.21.15.117:0/3783811867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c09a1e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa37c002f30 tx=0x7fa37c006dc0).ready entity=mon.1 client_cookie=aae608b61cf66319 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:43.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.308+0000 7fa38b7fe700 1 -- 172.21.15.117:0/3783811867 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37c0097f0 con 0x7fa38c060580 2020-11-20T13:55:43.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa38b7fe700 1 -- 172.21.15.117:0/3783811867 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c009950 con 0x7fa38c060580 2020-11-20T13:55:43.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa38b7fe700 1 -- 172.21.15.117:0/3783811867 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa37c009cb0 con 0x7fa38c060580 2020-11-20T13:55:43.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 -- 172.21.15.117:0/3783811867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 msgr2=0x7fa38c09a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:43.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 --2- 172.21.15.117:0/3783811867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c09a1e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa37c002f30 tx=0x7fa37c006dc0).stop 2020-11-20T13:55:43.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 -- 172.21.15.117:0/3783811867 shutdown_connections 2020-11-20T13:55:43.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 --2- 172.21.15.117:0/3783811867 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c0af850 0x7fa38c0b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 --2- 172.21.15.117:0/3783811867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c09a1e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 -- 172.21.15.117:0/3783811867 >> 172.21.15.117:0/3783811867 conn(0x7fa38c05f880 msgr2=0x7fa38c0ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:43.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 -- 172.21.15.117:0/3783811867 shutdown_connections 2020-11-20T13:55:43.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.309+0000 7fa390df4700 1 -- 172.21.15.117:0/3783811867 wait complete. 2020-11-20T13:55:43.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.310+0000 7fa390df4700 1 Processor -- start 2020-11-20T13:55:43.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.310+0000 7fa390df4700 1 -- start start 2020-11-20T13:55:43.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa390df4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c025950 0x7fa38c09bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa390df4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa390df4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa38c0af850 0x7fa38c07f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa390df4700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c158120 con 0x7fa38c0af850 2020-11-20T13:55:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa390df4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1583c0 con 0x7fa38c060580 2020-11-20T13:55:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa390df4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c13e080 con 0x7fa38c025950 2020-11-20T13:55:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39848/0 (socket says 172.21.15.117:39848) 2020-11-20T13:55:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 -- 172.21.15.117:0/877317198 learned_addr learned my addr 172.21.15.117:0/877317198 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa38bfff700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c025950 0x7fa38c09bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa392056700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa38c0af850 0x7fa38c07f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 -- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c025950 msgr2=0x7fa38c09bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c025950 0x7fa38c09bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 -- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa38c0af850 msgr2=0x7fa38c07f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa38c0af850 0x7fa38c07f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:43.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa383fff700 1 -- 172.21.15.117:0/877317198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c157c20 con 0x7fa38c060580 2020-11-20T13:55:43.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.311+0000 7fa392056700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa38c0af850 0x7fa38c07f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:43.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.312+0000 7fa383fff700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c157730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa374000e90 tx=0x7fa374007a90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:43.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.312+0000 7fa3897fa700 1 -- 172.21.15.117:0/877317198 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3740055e0 con 0x7fa38c060580 2020-11-20T13:55:43.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.312+0000 7fa3897fa700 1 -- 172.21.15.117:0/877317198 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa374005740 con 0x7fa38c060580 2020-11-20T13:55:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.312+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38c157e70 con 0x7fa38c060580 2020-11-20T13:55:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.312+0000 7fa3897fa700 1 -- 172.21.15.117:0/877317198 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa374005aa0 con 0x7fa38c060580 2020-11-20T13:55:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.312+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c07f950 con 0x7fa38c060580 2020-11-20T13:55:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.313+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38c057050 con 0x7fa38c060580 2020-11-20T13:55:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.313+0000 7fa3897fa700 1 -- 172.21.15.117:0/877317198 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa374005c00 con 0x7fa38c060580 2020-11-20T13:55:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.313+0000 7fa3897fa700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa370051320 0x7fa3700537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.313+0000 7fa3897fa700 1 -- 172.21.15.117:0/877317198 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5242+0+0 (secure 0 0 0) 0x7fa37406abf0 con 0x7fa38c060580 2020-11-20T13:55:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.313+0000 7fa38bfff700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa370051320 0x7fa3700537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:43.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.313+0000 7fa38bfff700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa370051320 0x7fa3700537e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa37c019000 tx=0x7fa37c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:43.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.316+0000 7fa3897fa700 1 -- 172.21.15.117:0/877317198 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa374043250 con 0x7fa38c060580 2020-11-20T13:55:43.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:43.472+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa38c0620f0 con 0x7fa38c060580 2020-11-20T13:55:44.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.188+0000 7fa3897fa700 1 -- 172.21.15.117:0/877317198 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa374046870 con 0x7fa38c060580 2020-11-20T13:55:44.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa370051320 msgr2=0x7fa3700537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa370051320 0x7fa3700537e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa37c019000 tx=0x7fa37c015000).stop 2020-11-20T13:55:44.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 msgr2=0x7fa38c157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c157730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa374000e90 tx=0x7fa374007a90).stop 2020-11-20T13:55:44.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 shutdown_connections 2020-11-20T13:55:44.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa370051320 0x7fa3700537e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa38c025950 0x7fa38c09bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa38c0af850 0x7fa38c07f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 --2- 172.21.15.117:0/877317198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa38c060580 0x7fa38c157730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 >> 172.21.15.117:0/877317198 conn(0x7fa38c05f880 msgr2=0x7fa38c0b9110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:44.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 shutdown_connections 2020-11-20T13:55:44.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.191+0000 7fa390df4700 1 -- 172.21.15.117:0/877317198 wait complete. 2020-11-20T13:55:44.193 INFO:teuthology.orchestra.run.smithi117.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-20T13:55:44.204 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-20T13:55:44.204 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T13:55:44.204 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T13:55:44.207 INFO:tasks.ceph.mds.a:Started 2020-11-20T13:55:44.207 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T13:55:44.207 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T13:55:44.209 INFO:tasks.ceph.mds.b:Started 2020-11-20T13:55:44.210 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-20T13:55:44.210 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:55:44.285 INFO:tasks.ceph.mds.a.smithi117.stdout:starting mds.a at 2020-11-20T13:55:44.290 INFO:tasks.ceph.mds.b.smithi085.stdout:starting mds.b at 2020-11-20T13:55:44.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.322+0000 7f6838558700 1 Processor -- start 2020-11-20T13:55:44.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.322+0000 7f6838558700 1 -- start start 2020-11-20T13:55:44.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6838558700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683005cd80 0x7f68300607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6838558700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f683009d2c0 0x7f68300aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6838558700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68300b5fc0 con 0x7f683009ce40 2020-11-20T13:55:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6838558700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6830060d20 con 0x7f683009d2c0 2020-11-20T13:55:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6838558700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b6260 con 0x7f683005cd80 2020-11-20T13:55:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6835af3700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f683009ce40 0x7f6830143620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59032/0 (socket says 172.21.15.117:59032) 2020-11-20T13:55:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6835af3700 1 -- 172.21.15.117:0/334522916 learned_addr learned my addr 172.21.15.117:0/334522916 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.323+0000 7f6836af5700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f683009d2c0 0x7f68300aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f68362f4700 1 --2- 172.21.15.117:0/334522916 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683005cd80 0x7f68300607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6836af5700 1 -- 172.21.15.117:0/334522916 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683005cd80 msgr2=0x7f68300607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6836af5700 1 --2- 172.21.15.117:0/334522916 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683005cd80 0x7f68300607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6836af5700 1 -- 172.21.15.117:0/334522916 >> v1:172.21.15.117:6789/0 conn(0x7f683009ce40 legacy=0x7f6830143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6836af5700 1 -- 172.21.15.117:0/334522916 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300b6500 con 0x7f683009d2c0 2020-11-20T13:55:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f68352f2700 1 -- 172.21.15.117:0/334522916 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71872935 0 0) 0x7f68300b5fc0 con 0x7f683009ce40 2020-11-20T13:55:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f68362f4700 1 --2- 172.21.15.117:0/334522916 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683005cd80 0x7f68300607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6836af5700 1 --2- 172.21.15.117:0/334522916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683009d2c0 0x7f68300aff60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f68280032b0 tx=0x7f68280091a0).ready entity=mon.1 client_cookie=abdf7f5a8cc8dd26 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f68352f2700 1 -- 172.21.15.117:0/334522916 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f682800ba50 con 0x7f683009d2c0 2020-11-20T13:55:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f68352f2700 1 -- 172.21.15.117:0/334522916 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800bbb0 con 0x7f683009d2c0 2020-11-20T13:55:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f68352f2700 1 -- 172.21.15.117:0/334522916 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f682800bd20 con 0x7f683009d2c0 2020-11-20T13:55:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6838558700 1 -- 172.21.15.117:0/334522916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683009d2c0 msgr2=0x7f68300aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6838558700 1 --2- 172.21.15.117:0/334522916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683009d2c0 0x7f68300aff60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f68280032b0 tx=0x7f68280091a0).stop 2020-11-20T13:55:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6838558700 1 -- 172.21.15.117:0/334522916 shutdown_connections 2020-11-20T13:55:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6838558700 1 --2- 172.21.15.117:0/334522916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683009d2c0 0x7f68300aff60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6838558700 1 --2- 172.21.15.117:0/334522916 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683005cd80 0x7f68300607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.324+0000 7f6838558700 1 -- 172.21.15.117:0/334522916 >> 172.21.15.117:0/334522916 conn(0x7f68300b7480 msgr2=0x7f6830062f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.325+0000 7f6838558700 1 -- 172.21.15.117:0/334522916 shutdown_connections 2020-11-20T13:55:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.325+0000 7f6838558700 1 -- 172.21.15.117:0/334522916 wait complete. 2020-11-20T13:55:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.325+0000 7f6838558700 1 Processor -- start 2020-11-20T13:55:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6838558700 1 -- start start 2020-11-20T13:55:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6838558700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683005cd80 0x7f68301429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6838558700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f683009ce40 0x7f68301557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6838558700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683009d2c0 0x7f6830155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6838558700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301565e0 con 0x7f683009ce40 2020-11-20T13:55:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6838558700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830156880 con 0x7f683005cd80 2020-11-20T13:55:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6838558700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6830156b20 con 0x7f683009d2c0 2020-11-20T13:55:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f68362f4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683005cd80 0x7f68301429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f68362f4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683005cd80 0x7f68301429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39878/0 (socket says 172.21.15.117:39878) 2020-11-20T13:55:44.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f68362f4700 1 -- 172.21.15.117:0/1259694090 learned_addr learned my addr 172.21.15.117:0/1259694090 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:44.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6836af5700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683009d2c0 0x7f6830155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.326+0000 7f6835af3700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f683009ce40 0x7f68301557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f68362f4700 1 -- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683009d2c0 msgr2=0x7f6830155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f68362f4700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683009d2c0 0x7f6830155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f68362f4700 1 -- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f683009ce40 msgr2=0x7f68301557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f68362f4700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f683009ce40 0x7f68301557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f68362f4700 1 -- 172.21.15.117:0/1259694090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68301395a0 con 0x7f683005cd80 2020-11-20T13:55:44.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f68362f4700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683005cd80 0x7f68301429e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f682c007000 tx=0x7f682c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f681f7fe700 1 -- 172.21.15.117:0/1259694090 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f682c00e510 con 0x7f683005cd80 2020-11-20T13:55:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f681f7fe700 1 -- 172.21.15.117:0/1259694090 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c00e670 con 0x7f683005cd80 2020-11-20T13:55:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301561c0 con 0x7f683005cd80 2020-11-20T13:55:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f681f7fe700 1 -- 172.21.15.117:0/1259694090 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f682c00e9d0 con 0x7f683005cd80 2020-11-20T13:55:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.327+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6830157200 con 0x7f683005cd80 2020-11-20T13:55:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.328+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6830156430 con 0x7f683005cd80 2020-11-20T13:55:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.328+0000 7f681f7fe700 1 -- 172.21.15.117:0/1259694090 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f682c019030 con 0x7f683005cd80 2020-11-20T13:55:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.328+0000 7f681f7fe700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f68100513c0 0x7f6810053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.329+0000 7f681f7fe700 1 -- 172.21.15.117:0/1259694090 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f682c0684e0 con 0x7f683005cd80 2020-11-20T13:55:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.329+0000 7f6835af3700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f68100513c0 0x7f6810053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.329+0000 7f6835af3700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f68100513c0 0x7f6810053880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f682000d000 tx=0x7f68200079d0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.332+0000 7f681f7fe700 1 -- 172.21.15.117:0/1259694090 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f682c040c40 con 0x7f683005cd80 2020-11-20T13:55:44.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.480+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6830156ce0 con 0x7f683005cd80 2020-11-20T13:55:44.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.481+0000 7f681f7fe700 1 -- 172.21.15.117:0/1259694090 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13239 (secure 0 0 0) 0x7f682c0243d0 con 0x7f683005cd80 2020-11-20T13:55:44.482 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:44.482 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":17,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:44.182174+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:44.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.482+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f68100513c0 msgr2=0x7f6810053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f68100513c0 0x7f6810053880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f682000d000 tx=0x7f68200079d0).stop 2020-11-20T13:55:44.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683005cd80 msgr2=0x7f68301429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683005cd80 0x7f68301429e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f682c007000 tx=0x7f682c008040).stop 2020-11-20T13:55:44.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 shutdown_connections 2020-11-20T13:55:44.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f68100513c0 0x7f6810053880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f683009d2c0 0x7f6830155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f683005cd80 0x7f68301429e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 --2- 172.21.15.117:0/1259694090 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f683009ce40 0x7f68301557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 >> 172.21.15.117:0/1259694090 conn(0x7f68300b7480 msgr2=0x7f683009d740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:44.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 shutdown_connections 2020-11-20T13:55:44.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.483+0000 7f6838558700 1 -- 172.21.15.117:0/1259694090 wait complete. 2020-11-20T13:55:44.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T13:55:44.494 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:55:44.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.602+0000 7f45b0774700 1 Processor -- start 2020-11-20T13:55:44.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.602+0000 7f45b0774700 1 -- start start 2020-11-20T13:55:44.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45b0774700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f45a809bd80 0x7f45a809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45b0774700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45a809a110 0x7f45a81437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45b0774700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a80999d0 con 0x7f45a8143c90 2020-11-20T13:55:44.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45b0774700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a8099c70 con 0x7f45a809bd80 2020-11-20T13:55:44.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45b0774700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a809c160 con 0x7f45a809a110 2020-11-20T13:55:44.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45add0f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45a809a110 0x7f45a81437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45add0f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45a809a110 0x7f45a81437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39884/0 (socket says 172.21.15.117:39884) 2020-11-20T13:55:44.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45add0f700 1 -- 172.21.15.117:0/3393233819 learned_addr learned my addr 172.21.15.117:0/3393233819 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:44.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45ae510700 1 --2- 172.21.15.117:0/3393233819 >> v2:172.21.15.85:3300/0 conn(0x7f45a809bd80 0x7f45a809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45add0f700 1 -- 172.21.15.117:0/3393233819 >> v2:172.21.15.85:3300/0 conn(0x7f45a809bd80 msgr2=0x7f45a809cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:44.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45add0f700 1 --2- 172.21.15.117:0/3393233819 >> v2:172.21.15.85:3300/0 conn(0x7f45a809bd80 0x7f45a809cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45add0f700 1 -- 172.21.15.117:0/3393233819 >> v1:172.21.15.85:6789/0 conn(0x7f45a8143c90 legacy=0x7f45a80b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:44.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.603+0000 7f45add0f700 1 -- 172.21.15.117:0/3393233819 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a81440e0 con 0x7f45a809a110 2020-11-20T13:55:44.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45add0f700 1 --2- 172.21.15.117:0/3393233819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a809a110 0x7f45a81437a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f45a4002f30 tx=0x7f45a4006e80).ready entity=mon.1 client_cookie=7986c34296e13d9f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45ad50e700 1 -- 172.21.15.117:0/3393233819 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a40097f0 con 0x7f45a809a110 2020-11-20T13:55:44.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45ad50e700 1 -- 172.21.15.117:0/3393233819 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4009950 con 0x7f45a809a110 2020-11-20T13:55:44.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45ad50e700 1 -- 172.21.15.117:0/3393233819 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a4009cb0 con 0x7f45a809a110 2020-11-20T13:55:44.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 -- 172.21.15.117:0/3393233819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a809a110 msgr2=0x7f45a81437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 --2- 172.21.15.117:0/3393233819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a809a110 0x7f45a81437a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f45a4002f30 tx=0x7f45a4006e80).stop 2020-11-20T13:55:44.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 -- 172.21.15.117:0/3393233819 shutdown_connections 2020-11-20T13:55:44.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 --2- 172.21.15.117:0/3393233819 >> v2:172.21.15.85:3300/0 conn(0x7f45a809bd80 0x7f45a809cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 --2- 172.21.15.117:0/3393233819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a809a110 0x7f45a81437a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 -- 172.21.15.117:0/3393233819 >> 172.21.15.117:0/3393233819 conn(0x7f45a80b9770 msgr2=0x7f45a80b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:44.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 -- 172.21.15.117:0/3393233819 shutdown_connections 2020-11-20T13:55:44.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.604+0000 7f45b0774700 1 -- 172.21.15.117:0/3393233819 wait complete. 2020-11-20T13:55:44.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.605+0000 7f45b0774700 1 Processor -- start 2020-11-20T13:55:44.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.605+0000 7f45b0774700 1 -- start start 2020-11-20T13:55:44.614 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45b0774700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45a809a110 0x7f45a81384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.614 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45b0774700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45a809bd80 0x7f45a813b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.614 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45b0774700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a8143c90 0x7f45a8139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.614 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45b0774700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81389d0 con 0x7f45a809bd80 2020-11-20T13:55:44.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45b0774700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a813b940 con 0x7f45a8143c90 2020-11-20T13:55:44.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45b0774700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a813bbd0 con 0x7f45a809a110 2020-11-20T13:55:44.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a8143c90 0x7f45a8139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a8143c90 0x7f45a8139dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39894/0 (socket says 172.21.15.117:39894) 2020-11-20T13:55:44.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45add0f700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45a809bd80 0x7f45a813b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 -- 172.21.15.117:0/1620272612 learned_addr learned my addr 172.21.15.117:0/1620272612 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45ae510700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45a809a110 0x7f45a81384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 -- 172.21.15.117:0/1620272612 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45a809a110 msgr2=0x7f45a81384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45a809a110 0x7f45a81384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 -- 172.21.15.117:0/1620272612 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45a809bd80 msgr2=0x7f45a813b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45a809bd80 0x7f45a813b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 -- 172.21.15.117:0/1620272612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a813be70 con 0x7f45a8143c90 2020-11-20T13:55:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45aed11700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a8143c90 0x7f45a8139dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f45a809afe0 tx=0x7f45a0007e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.606+0000 7f45977fe700 1 -- 172.21.15.117:0/1620272612 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a000a5a0 con 0x7f45a8143c90 2020-11-20T13:55:44.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.607+0000 7f45977fe700 1 -- 172.21.15.117:0/1620272612 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a000a700 con 0x7f45a8143c90 2020-11-20T13:55:44.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.607+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a8066aa0 con 0x7f45a8143c90 2020-11-20T13:55:44.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.607+0000 7f45977fe700 1 -- 172.21.15.117:0/1620272612 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a000aa60 con 0x7f45a8143c90 2020-11-20T13:55:44.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.607+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a8064790 con 0x7f45a8143c90 2020-11-20T13:55:44.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.608+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a8057050 con 0x7f45a8143c90 2020-11-20T13:55:44.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.608+0000 7f45977fe700 1 -- 172.21.15.117:0/1620272612 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f45a0019030 con 0x7f45a8143c90 2020-11-20T13:55:44.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.608+0000 7f45977fe700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.608+0000 7f45977fe700 1 -- 172.21.15.117:0/1620272612 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f45a0068500 con 0x7f45a8143c90 2020-11-20T13:55:44.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.608+0000 7f45ae510700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.608+0000 7f45ae510700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f459800c000 tx=0x7f4598008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.611+0000 7f45977fe700 1 -- 172.21.15.117:0/1620272612 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f45a0040bb0 con 0x7f45a8143c90 2020-11-20T13:55:44.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.758+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f45a805ca00 con 0x7f45a8143c90 2020-11-20T13:55:44.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.759+0000 7f45977fe700 1 -- 172.21.15.117:0/1620272612 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f45a00441d0 con 0x7f45a8143c90 2020-11-20T13:55:44.759 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:55:44.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 msgr2=0x7f45880536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f459800c000 tx=0x7f4598008040).stop 2020-11-20T13:55:44.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a8143c90 msgr2=0x7f45a8139dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a8143c90 0x7f45a8139dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f45a809afe0 tx=0x7f45a0007e80).stop 2020-11-20T13:55:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 shutdown_connections 2020-11-20T13:55:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45a809a110 0x7f45a81384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45a8143c90 0x7f45a8139dd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 --2- 172.21.15.117:0/1620272612 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45a809bd80 0x7f45a813b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 >> 172.21.15.117:0/1620272612 conn(0x7f45a80b9770 msgr2=0x7f45a809f090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:44.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 shutdown_connections 2020-11-20T13:55:44.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.760+0000 7f45b0774700 1 -- 172.21.15.117:0/1620272612 wait complete. 2020-11-20T13:55:44.771 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:55:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.878+0000 7fc331b00700 1 Processor -- start 2020-11-20T13:55:44.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.878+0000 7fc331b00700 1 -- start start 2020-11-20T13:55:44.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc331b00700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fc32c098d60 0x7fc32c09b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc331b00700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc32c0b5240 0x7fc32c0630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc331b00700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c0b4860 con 0x7fc32c0635b0 2020-11-20T13:55:44.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc331b00700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c1439f0 con 0x7fc32c098d60 2020-11-20T13:55:44.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc331b00700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c0b4b00 con 0x7fc32c0b5240 2020-11-20T13:55:44.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc32affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc32c0b5240 0x7fc32c0630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc32affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc32c0b5240 0x7fc32c0630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39900/0 (socket says 172.21.15.117:39900) 2020-11-20T13:55:44.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc32affd700 1 -- 172.21.15.117:0/2043166492 learned_addr learned my addr 172.21.15.117:0/2043166492 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:44.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.879+0000 7fc32b7fe700 1 --2- 172.21.15.117:0/2043166492 >> v2:172.21.15.85:3300/0 conn(0x7fc32c098d60 0x7fc32c09b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32affd700 1 -- 172.21.15.117:0/2043166492 >> v2:172.21.15.85:3300/0 conn(0x7fc32c098d60 msgr2=0x7fc32c09b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:44.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32affd700 1 --2- 172.21.15.117:0/2043166492 >> v2:172.21.15.85:3300/0 conn(0x7fc32c098d60 0x7fc32c09b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32affd700 1 -- 172.21.15.117:0/2043166492 >> v1:172.21.15.85:6789/0 conn(0x7fc32c0635b0 legacy=0x7fc32c09c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:44.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32affd700 1 -- 172.21.15.117:0/2043166492 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0b4da0 con 0x7fc32c0b5240 2020-11-20T13:55:44.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32affd700 1 --2- 172.21.15.117:0/2043166492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0b5240 0x7fc32c0630c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc32c09b040 tx=0x7fc320006e80).ready entity=mon.1 client_cookie=75d977b0591daba6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32a7fc700 1 -- 172.21.15.117:0/2043166492 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3200097b0 con 0x7fc32c0b5240 2020-11-20T13:55:44.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32a7fc700 1 -- 172.21.15.117:0/2043166492 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320009910 con 0x7fc32c0b5240 2020-11-20T13:55:44.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc32a7fc700 1 -- 172.21.15.117:0/2043166492 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc320009a80 con 0x7fc32c0b5240 2020-11-20T13:55:44.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 -- 172.21.15.117:0/2043166492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0b5240 msgr2=0x7fc32c0630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 --2- 172.21.15.117:0/2043166492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0b5240 0x7fc32c0630c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fc32c09b040 tx=0x7fc320006e80).stop 2020-11-20T13:55:44.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 -- 172.21.15.117:0/2043166492 shutdown_connections 2020-11-20T13:55:44.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 --2- 172.21.15.117:0/2043166492 >> v2:172.21.15.85:3300/0 conn(0x7fc32c098d60 0x7fc32c09b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 --2- 172.21.15.117:0/2043166492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0b5240 0x7fc32c0630c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 -- 172.21.15.117:0/2043166492 >> 172.21.15.117:0/2043166492 conn(0x7fc32c0b9770 msgr2=0x7fc32c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:44.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 -- 172.21.15.117:0/2043166492 shutdown_connections 2020-11-20T13:55:44.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.880+0000 7fc331b00700 1 -- 172.21.15.117:0/2043166492 wait complete. 2020-11-20T13:55:44.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.881+0000 7fc331b00700 1 Processor -- start 2020-11-20T13:55:44.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc331b00700 1 -- start start 2020-11-20T13:55:44.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc331b00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0635b0 0x7fc32c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc331b00700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc32c098d60 0x7fc32c065450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc331b00700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc32c0b5240 0x7fc32c1415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc331b00700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0b26f0 con 0x7fc32c098d60 2020-11-20T13:55:44.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc331b00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0b2990 con 0x7fc32c0635b0 2020-11-20T13:55:44.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc331b00700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0b2c30 con 0x7fc32c0b5240 2020-11-20T13:55:44.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0635b0 0x7fc32c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0635b0 0x7fc32c0b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39904/0 (socket says 172.21.15.117:39904) 2020-11-20T13:55:44.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32b7fe700 1 -- 172.21.15.117:0/4192154042 learned_addr learned my addr 172.21.15.117:0/4192154042 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:44.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32affd700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc32c098d60 0x7fc32c065450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32bfff700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc32c0b5240 0x7fc32c1415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32b7fe700 1 -- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc32c0b5240 msgr2=0x7fc32c1415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32b7fe700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc32c0b5240 0x7fc32c1415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32b7fe700 1 -- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc32c098d60 msgr2=0x7fc32c065450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:44.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.882+0000 7fc32b7fe700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc32c098d60 0x7fc32c065450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:44.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.883+0000 7fc32b7fe700 1 -- 172.21.15.117:0/4192154042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0b2300 con 0x7fc32c0635b0 2020-11-20T13:55:44.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.883+0000 7fc32b7fe700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0635b0 0x7fc32c0b30d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fc31c007000 tx=0x7fc31c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.883+0000 7fc328ff9700 1 -- 172.21.15.117:0/4192154042 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c00e5e0 con 0x7fc32c0635b0 2020-11-20T13:55:44.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.883+0000 7fc328ff9700 1 -- 172.21.15.117:0/4192154042 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00c030 con 0x7fc32c0635b0 2020-11-20T13:55:44.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.883+0000 7fc328ff9700 1 -- 172.21.15.117:0/4192154042 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc31c00e930 con 0x7fc32c0635b0 2020-11-20T13:55:44.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.883+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c0b0d30 con 0x7fc32c0635b0 2020-11-20T13:55:44.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.883+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c155920 con 0x7fc32c0635b0 2020-11-20T13:55:44.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.884+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c0b2550 con 0x7fc32c0635b0 2020-11-20T13:55:44.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.884+0000 7fc328ff9700 1 -- 172.21.15.117:0/4192154042 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc31c019030 con 0x7fc32c0635b0 2020-11-20T13:55:44.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.884+0000 7fc328ff9700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc30c0513e0 0x7fc30c0538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:44.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.884+0000 7fc328ff9700 1 -- 172.21.15.117:0/4192154042 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc31c068420 con 0x7fc32c0635b0 2020-11-20T13:55:44.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.884+0000 7fc32affd700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc30c0513e0 0x7fc30c0538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:44.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.885+0000 7fc32affd700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc30c0513e0 0x7fc30c0538a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc32001c000 tx=0x7fc320016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:44.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:44.888+0000 7fc328ff9700 1 -- 172.21.15.117:0/4192154042 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc32c0b2550 con 0x7fc32c0635b0 2020-11-20T13:55:45.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.040+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc32c0b2550 con 0x7fc32c0635b0 2020-11-20T13:55:45.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.040+0000 7fc328ff9700 1 -- 172.21.15.117:0/4192154042 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc32c0b2550 con 0x7fc32c0635b0 2020-11-20T13:55:45.041 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:55:45.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc30c0513e0 msgr2=0x7fc30c0538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc30c0513e0 0x7fc30c0538a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fc32001c000 tx=0x7fc320016000).stop 2020-11-20T13:55:45.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0635b0 msgr2=0x7fc32c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0635b0 0x7fc32c0b30d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fc31c007000 tx=0x7fc31c008040).stop 2020-11-20T13:55:45.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 shutdown_connections 2020-11-20T13:55:45.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc30c0513e0 0x7fc30c0538a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc32c0b5240 0x7fc32c1415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32c0635b0 0x7fc32c0b30d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 --2- 172.21.15.117:0/4192154042 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc32c098d60 0x7fc32c065450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 >> 172.21.15.117:0/4192154042 conn(0x7fc32c0b9770 msgr2=0x7fc32c062750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:45.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 shutdown_connections 2020-11-20T13:55:45.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.042+0000 7fc331b00700 1 -- 172.21.15.117:0/4192154042 wait complete. 2020-11-20T13:55:45.052 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-20T13:55:45.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.158+0000 7fa662c5c700 1 Processor -- start 2020-11-20T13:55:45.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.158+0000 7fa662c5c700 1 -- start start 2020-11-20T13:55:45.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa662c5c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fa65c098e30 0x7fa65c09b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa662c5c700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c0b5510 con 0x7fa65c09b700 2020-11-20T13:55:45.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa662c5c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c0b57b0 con 0x7fa65c0ae6f0 2020-11-20T13:55:45.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa662c5c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c0b5a50 con 0x7fa65c098e30 2020-11-20T13:55:45.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa6611f9700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa65c0ae6f0 0x7fa65c0aeb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59076/0 (socket says 172.21.15.117:59076) 2020-11-20T13:55:45.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa6611f9700 1 -- 172.21.15.117:0/4193681007 learned_addr learned my addr 172.21.15.117:0/4193681007 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:45.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777996707 0 0) 0x7fa65c0b57b0 con 0x7fa65c0ae6f0 2020-11-20T13:55:45.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa6609f8700 1 --2- 172.21.15.117:0/4193681007 >> v2:172.21.15.85:3300/0 conn(0x7fa65c098e30 0x7fa65c09b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa648003230 con 0x7fa65c0ae6f0 2020-11-20T13:55:45.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1541842570 0 0) 0x7fa648003230 con 0x7fa65c0ae6f0 2020-11-20T13:55:45.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 >> v2:172.21.15.85:3300/0 conn(0x7fa65c098e30 msgr2=0x7fa65c09b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:45.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 --2- 172.21.15.117:0/4193681007 >> v2:172.21.15.85:3300/0 conn(0x7fa65c098e30 0x7fa65c09b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 >> v1:172.21.15.85:6789/0 conn(0x7fa65c09b700 legacy=0x7fa65c1438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c099210 con 0x7fa65c0ae6f0 2020-11-20T13:55:45.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fa64c001710 con 0x7fa65c0ae6f0 2020-11-20T13:55:45.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 >> v1:172.21.15.117:6789/0 conn(0x7fa65c0ae6f0 legacy=0x7fa65c0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa648003720 0x7fa648003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6480044d0 0x7fa648004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa648004e40 0x7fa64800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0b5a50 con 0x7fa648004e40 2020-11-20T13:55:45.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa648004340 con 0x7fa6480044d0 2020-11-20T13:55:45.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64800bc40 con 0x7fa648003720 2020-11-20T13:55:45.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.159+0000 7fa65bfff700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6480044d0 0x7fa648004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa6609f8700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa648003720 0x7fa648003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa6611f9700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa648004e40 0x7fa64800b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65bfff700 1 -- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa648003720 msgr2=0x7fa648003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65bfff700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa648003720 0x7fa648003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65bfff700 1 -- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa648004e40 msgr2=0x7fa64800b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65bfff700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa648004e40 0x7fa64800b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa6611f9700 1 -- 172.21.15.117:0/4193681007 reap_dead start 2020-11-20T13:55:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65bfff700 1 -- 172.21.15.117:0/4193681007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648003230 con 0x7fa6480044d0 2020-11-20T13:55:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65bfff700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6480044d0 0x7fa648004950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa65c060b70 tx=0x7fa650007b60).ready entity=mon.1 client_cookie=af3319184a9f8b7c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa65000c660 con 0x7fa6480044d0 2020-11-20T13:55:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65000c7c0 con 0x7fa6480044d0 2020-11-20T13:55:45.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa65b7fe700 1 -- 172.21.15.117:0/4193681007 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa65000cb20 con 0x7fa6480044d0 2020-11-20T13:55:45.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa662c5c700 1 -- 172.21.15.117:0/4193681007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6480044d0 msgr2=0x7fa648004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.160+0000 7fa662c5c700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6480044d0 0x7fa648004950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa65c060b70 tx=0x7fa650007b60).stop 2020-11-20T13:55:45.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.161+0000 7fa662c5c700 1 -- 172.21.15.117:0/4193681007 shutdown_connections 2020-11-20T13:55:45.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.161+0000 7fa662c5c700 1 --2- 172.21.15.117:0/4193681007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6480044d0 0x7fa648004950 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.161+0000 7fa662c5c700 1 -- 172.21.15.117:0/4193681007 >> 172.21.15.117:0/4193681007 conn(0x7fa65c05b9f0 msgr2=0x7fa65c05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:45.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.161+0000 7fa662c5c700 1 -- 172.21.15.117:0/4193681007 shutdown_connections 2020-11-20T13:55:45.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.161+0000 7fa662c5c700 1 -- 172.21.15.117:0/4193681007 wait complete. 2020-11-20T13:55:45.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 Processor -- start 2020-11-20T13:55:45.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 -- start start 2020-11-20T13:55:45.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa6480044d0 0x7fa65c09d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c146a10 0x7fa65c15a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa65c066f20 0x7fa65c140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c145240 con 0x7fa65c066f20 2020-11-20T13:55:45.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0673a0 con 0x7fa65c146a10 2020-11-20T13:55:45.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.162+0000 7fa662c5c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c147040 con 0x7fa6480044d0 2020-11-20T13:55:45.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c146a10 0x7fa65c15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c146a10 0x7fa65c15a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39926/0 (socket says 172.21.15.117:39926) 2020-11-20T13:55:45.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 -- 172.21.15.117:0/631504177 learned_addr learned my addr 172.21.15.117:0/631504177 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:45.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa6609f8700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa6480044d0 0x7fa65c09d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa6611f9700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa65c066f20 0x7fa65c140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 -- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa6480044d0 msgr2=0x7fa65c09d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa6480044d0 0x7fa65c09d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 -- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa65c066f20 msgr2=0x7fa65c140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa65c066f20 0x7fa65c140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 -- 172.21.15.117:0/631504177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c1472e0 con 0x7fa65c146a10 2020-11-20T13:55:45.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa65bfff700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c146a10 0x7fa65c15a060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa65c145510 tx=0x7fa650016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:45.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa659ffb700 1 -- 172.21.15.117:0/631504177 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa650013d20 con 0x7fa65c146a10 2020-11-20T13:55:45.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa659ffb700 1 -- 172.21.15.117:0/631504177 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa650013e80 con 0x7fa65c146a10 2020-11-20T13:55:45.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c140d10 con 0x7fa65c146a10 2020-11-20T13:55:45.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa659ffb700 1 -- 172.21.15.117:0/631504177 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa650007220 con 0x7fa65c146a10 2020-11-20T13:55:45.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.163+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c145630 con 0x7fa65c146a10 2020-11-20T13:55:45.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.164+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c146e90 con 0x7fa65c146a10 2020-11-20T13:55:45.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.164+0000 7fa659ffb700 1 -- 172.21.15.117:0/631504177 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa650024030 con 0x7fa65c146a10 2020-11-20T13:55:45.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.164+0000 7fa659ffb700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa63c051290 0x7fa63c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.164+0000 7fa659ffb700 1 -- 172.21.15.117:0/631504177 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa650069a50 con 0x7fa65c146a10 2020-11-20T13:55:45.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.165+0000 7fa6609f8700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa63c051290 0x7fa63c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.165+0000 7fa6609f8700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa63c051290 0x7fa63c053750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa65400b000 tx=0x7fa6540091a0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:45.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.168+0000 7fa659ffb700 1 -- 172.21.15.117:0/631504177 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa650042100 con 0x7fa65c146a10 2020-11-20T13:55:45.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.323+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa65c057050 con 0x7fa65c146a10 2020-11-20T13:55:45.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.323+0000 7fa659ffb700 1 -- 172.21.15.117:0/631504177 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa650022070 con 0x7fa65c146a10 2020-11-20T13:55:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:45.324 INFO:teuthology.orchestra.run.smithi117.stdout:[ 2020-11-20T13:55:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:55:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "poolnum": 1, 2020-11-20T13:55:45.324 INFO:teuthology.orchestra.run.smithi117.stdout: "poolname": "device_health_metrics" 2020-11-20T13:55:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:55:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:55:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "poolnum": 2, 2020-11-20T13:55:45.325 INFO:teuthology.orchestra.run.smithi117.stdout: "poolname": "rbd" 2020-11-20T13:55:45.326 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:55:45.326 INFO:teuthology.orchestra.run.smithi117.stdout:] 2020-11-20T13:55:45.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.324+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa63c051290 msgr2=0x7fa63c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.324+0000 7fa662c5c700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa63c051290 0x7fa63c053750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fa65400b000 tx=0x7fa6540091a0).stop 2020-11-20T13:55:45.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.324+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c146a10 msgr2=0x7fa65c15a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.324+0000 7fa662c5c700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c146a10 0x7fa65c15a060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa65c145510 tx=0x7fa650016000).stop 2020-11-20T13:55:45.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 shutdown_connections 2020-11-20T13:55:45.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa63c051290 0x7fa63c053750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa6480044d0 0x7fa65c09d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa65c066f20 0x7fa65c140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 --2- 172.21.15.117:0/631504177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa65c146a10 0x7fa65c15a060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 >> 172.21.15.117:0/631504177 conn(0x7fa65c05b9f0 msgr2=0x7fa65c0b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:45.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 shutdown_connections 2020-11-20T13:55:45.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.325+0000 7fa662c5c700 1 -- 172.21.15.117:0/631504177 wait complete. 2020-11-20T13:55:45.335 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T13:55:45.336 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-20T13:55:45.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.445+0000 7fe56df32700 1 Processor -- start 2020-11-20T13:55:45.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.445+0000 7fe56df32700 1 -- start start 2020-11-20T13:55:45.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe56df32700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe568098d60 0x7fe56809b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe56df32700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b5240 0x7fe56809c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe56df32700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe5680b56d0 0x7fe56809cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe56df32700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5681439f0 con 0x7fe5680b56d0 2020-11-20T13:55:45.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe56df32700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56809d120 con 0x7fe568098d60 2020-11-20T13:55:45.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe56df32700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b4860 con 0x7fe5680b5240 2020-11-20T13:55:45.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe568098d60 0x7fe56809b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe568098d60 0x7fe56809b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39932/0 (socket says 172.21.15.117:39932) 2020-11-20T13:55:45.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 -- 172.21.15.117:0/3410995369 learned_addr learned my addr 172.21.15.117:0/3410995369 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:45.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 -- 172.21.15.117:0/3410995369 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b5240 msgr2=0x7fe56809c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:45.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 --2- 172.21.15.117:0/3410995369 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b5240 0x7fe56809c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:45.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 -- 172.21.15.117:0/3410995369 >> v2:172.21.15.85:3300/0 conn(0x7fe5680b56d0 msgr2=0x7fe56809cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:45.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe567fff700 1 --2- 172.21.15.117:0/3410995369 >> v2:172.21.15.85:3300/0 conn(0x7fe5680b56d0 0x7fe56809cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 --2- 172.21.15.117:0/3410995369 >> v2:172.21.15.85:3300/0 conn(0x7fe5680b56d0 0x7fe56809cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.446+0000 7fe5677fe700 1 -- 172.21.15.117:0/3410995369 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680b4b00 con 0x7fe568098d60 2020-11-20T13:55:45.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe5677fe700 1 --2- 172.21.15.117:0/3410995369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe56809b130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe558002f30 tx=0x7fe558006e80).ready entity=mon.1 client_cookie=44adf2185d2b8ffb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:45.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe5667fc700 1 -- 172.21.15.117:0/3410995369 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5580097f0 con 0x7fe568098d60 2020-11-20T13:55:45.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe5667fc700 1 -- 172.21.15.117:0/3410995369 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558009950 con 0x7fe568098d60 2020-11-20T13:55:45.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe5667fc700 1 -- 172.21.15.117:0/3410995369 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe558009cb0 con 0x7fe568098d60 2020-11-20T13:55:45.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 -- 172.21.15.117:0/3410995369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 msgr2=0x7fe56809b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 --2- 172.21.15.117:0/3410995369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe56809b130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe558002f30 tx=0x7fe558006e80).stop 2020-11-20T13:55:45.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 -- 172.21.15.117:0/3410995369 shutdown_connections 2020-11-20T13:55:45.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 --2- 172.21.15.117:0/3410995369 >> v2:172.21.15.85:3300/0 conn(0x7fe5680b56d0 0x7fe56809cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 --2- 172.21.15.117:0/3410995369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe56809b130 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 --2- 172.21.15.117:0/3410995369 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b5240 0x7fe56809c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:45.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 -- 172.21.15.117:0/3410995369 >> 172.21.15.117:0/3410995369 conn(0x7fe5680b9770 msgr2=0x7fe5680b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:45.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 -- 172.21.15.117:0/3410995369 shutdown_connections 2020-11-20T13:55:45.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.447+0000 7fe56df32700 1 -- 172.21.15.117:0/3410995369 wait complete. 2020-11-20T13:55:45.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.448+0000 7fe56df32700 1 Processor -- start 2020-11-20T13:55:45.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.448+0000 7fe56df32700 1 -- start start 2020-11-20T13:55:45.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe56df32700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe568154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe56df32700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5680b5240 0x7fe5680b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe56df32700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b56d0 0x7fe568142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe56df32700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b30b0 con 0x7fe5680b5240 2020-11-20T13:55:45.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe56df32700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b3350 con 0x7fe568098d60 2020-11-20T13:55:45.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe56df32700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b1600 con 0x7fe5680b56d0 2020-11-20T13:55:45.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe568154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe568154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39938/0 (socket says 172.21.15.117:39938) 2020-11-20T13:55:45.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 -- 172.21.15.117:0/3145916283 learned_addr learned my addr 172.21.15.117:0/3145916283 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:45.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe567fff700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b56d0 0x7fe568142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe566ffd700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5680b5240 0x7fe5680b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 -- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b56d0 msgr2=0x7fe568142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b56d0 0x7fe568142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 -- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5680b5240 msgr2=0x7fe5680b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:45.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5680b5240 0x7fe5680b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:45.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 -- 172.21.15.117:0/3145916283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680b17c0 con 0x7fe568098d60 2020-11-20T13:55:45.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe566ffd700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5680b5240 0x7fe5680b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:45.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.449+0000 7fe5677fe700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe568154ec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe558002f30 tx=0x7fe558015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:45.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.450+0000 7fe564ff9700 1 -- 172.21.15.117:0/3145916283 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe558011db0 con 0x7fe568098d60 2020-11-20T13:55:45.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.450+0000 7fe564ff9700 1 -- 172.21.15.117:0/3145916283 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55801e460 con 0x7fe568098d60 2020-11-20T13:55:45.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.450+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5680b1a10 con 0x7fe568098d60 2020-11-20T13:55:45.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.450+0000 7fe564ff9700 1 -- 172.21.15.117:0/3145916283 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55801e7c0 con 0x7fe568098d60 2020-11-20T13:55:45.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.450+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56813d910 con 0x7fe568098d60 2020-11-20T13:55:45.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.451+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5680b2f00 con 0x7fe568098d60 2020-11-20T13:55:45.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.451+0000 7fe564ff9700 1 -- 172.21.15.117:0/3145916283 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe558025030 con 0x7fe568098d60 2020-11-20T13:55:45.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.451+0000 7fe564ff9700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe548051290 0x7fe548053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:45.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.451+0000 7fe564ff9700 1 -- 172.21.15.117:0/3145916283 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe55806a170 con 0x7fe568098d60 2020-11-20T13:55:45.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.451+0000 7fe566ffd700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe548051290 0x7fe548053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:45.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.451+0000 7fe566ffd700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe548051290 0x7fe548053750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe55c007000 tx=0x7fe55c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:45.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.454+0000 7fe564ff9700 1 -- 172.21.15.117:0/3145916283 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe5580428a0 con 0x7fe568098d60 2020-11-20T13:55:45.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:45.605+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fe568057050 con 0x7fe568098d60 2020-11-20T13:55:46.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.307+0000 7fe564ff9700 1 -- 172.21.15.117:0/3145916283 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe5580230a0 con 0x7fe568098d60 2020-11-20T13:55:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.310+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe548051290 msgr2=0x7fe548053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.310+0000 7fe56df32700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe548051290 0x7fe548053750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe55c007000 tx=0x7fe55c008040).stop 2020-11-20T13:55:46.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.310+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 msgr2=0x7fe568154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:46.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.310+0000 7fe56df32700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe568154ec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fe558002f30 tx=0x7fe558015000).stop 2020-11-20T13:55:46.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 shutdown_connections 2020-11-20T13:55:46.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe548051290 0x7fe548053750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe5680b56d0 0x7fe568142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe568098d60 0x7fe568154ec0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 --2- 172.21.15.117:0/3145916283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5680b5240 0x7fe5680b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 >> 172.21.15.117:0/3145916283 conn(0x7fe5680b9770 msgr2=0x7fe5680b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:46.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 shutdown_connections 2020-11-20T13:55:46.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.311+0000 7fe56df32700 1 -- 172.21.15.117:0/3145916283 wait complete. 2020-11-20T13:55:46.314 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' created 2020-11-20T13:55:46.324 INFO:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2020-11-20T13:55:46.324 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd erasure-code-profile set cephfs_data m=2 k=2 crush-failure-domain=osd 2020-11-20T13:55:46.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb32ef75700 1 Processor -- start 2020-11-20T13:55:46.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb32ef75700 1 -- start start 2020-11-20T13:55:46.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb32ef75700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 0x7fb32809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb32ef75700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb328143eb0 0x7fb3280b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb32ef75700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3280999d0 con 0x7fb32809a110 2020-11-20T13:55:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb32ef75700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb328099c70 con 0x7fb328143eb0 2020-11-20T13:55:46.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb32ef75700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32809c160 con 0x7fb32809bd80 2020-11-20T13:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb327fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb32809a110 0x7fb3281437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59108/0 (socket says 172.21.15.117:59108) 2020-11-20T13:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.439+0000 7fb327fff700 1 -- 172.21.15.117:0/3752900033 learned_addr learned my addr 172.21.15.117:0/3752900033 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 0x7fb32809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682692669 0 0) 0x7fb3280999d0 con 0x7fb32809a110 2020-11-20T13:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb314003230 con 0x7fb32809a110 2020-11-20T13:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32d512700 1 --2- 172.21.15.117:0/3752900033 >> v2:172.21.15.85:3300/0 conn(0x7fb328143eb0 0x7fb3280b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 507807733 0 0) 0x7fb314003230 con 0x7fb32809a110 2020-11-20T13:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 msgr2=0x7fb32809cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 0x7fb32809cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 >> v2:172.21.15.85:3300/0 conn(0x7fb328143eb0 msgr2=0x7fb3280b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 --2- 172.21.15.117:0/3752900033 >> v2:172.21.15.85:3300/0 conn(0x7fb328143eb0 0x7fb3280b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328098cb0 con 0x7fb32809a110 2020-11-20T13:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fb31c0016d0 con 0x7fb32809a110 2020-11-20T13:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 >> v1:172.21.15.117:6789/0 conn(0x7fb32809a110 legacy=0x7fb3281437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb314003720 0x7fb314003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb3140044d0 0x7fb314004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 0x7fb314004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb328099c70 con 0x7fb3140044d0 2020-11-20T13:55:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32809c160 con 0x7fb314003720 2020-11-20T13:55:46.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb314004260 con 0x7fb32809bd80 2020-11-20T13:55:46.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb314003720 0x7fb314003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb327fff700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb3140044d0 0x7fb314004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32d512700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 0x7fb314004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 -- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 msgr2=0x7fb314004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb32809bd80 0x7fb314004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 -- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb3140044d0 msgr2=0x7fb314004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb3140044d0 0x7fb314004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 -- 172.21.15.117:0/3752900033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314003230 con 0x7fb314003720 2020-11-20T13:55:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.440+0000 7fb32cd11700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb314003720 0x7fb314003b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb31800c000 tx=0x7fb318008040).ready entity=mon.1 client_cookie=31c8ccff24f27513 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb31800e970 con 0x7fb314003720 2020-11-20T13:55:46.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31800ead0 con 0x7fb314003720 2020-11-20T13:55:46.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb3277fe700 1 -- 172.21.15.117:0/3752900033 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb31800ee30 con 0x7fb314003720 2020-11-20T13:55:46.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb32ef75700 1 -- 172.21.15.117:0/3752900033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb314003720 msgr2=0x7fb314003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:46.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb32ef75700 1 --2- 172.21.15.117:0/3752900033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb314003720 0x7fb314003b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fb31800c000 tx=0x7fb318008040).stop 2020-11-20T13:55:46.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb32d512700 1 -- 172.21.15.117:0/3752900033 reap_dead start 2020-11-20T13:55:46.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb32ef75700 1 -- 172.21.15.117:0/3752900033 shutdown_connections 2020-11-20T13:55:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb32ef75700 1 -- 172.21.15.117:0/3752900033 >> 172.21.15.117:0/3752900033 conn(0x7fb3280b9770 msgr2=0x7fb3280b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb32ef75700 1 -- 172.21.15.117:0/3752900033 shutdown_connections 2020-11-20T13:55:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.441+0000 7fb32ef75700 1 -- 172.21.15.117:0/3752900033 wait complete. 2020-11-20T13:55:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.442+0000 7fb32ef75700 1 Processor -- start 2020-11-20T13:55:46.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.442+0000 7fb32ef75700 1 -- start start 2020-11-20T13:55:46.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32ef75700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb328142d30 0x7fb328156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32ef75700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb32813c560 0x7fb328159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32ef75700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb328139330 0x7fb3281397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32ef75700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281560a0 con 0x7fb328142d30 2020-11-20T13:55:46.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32ef75700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb328156340 con 0x7fb32813c560 2020-11-20T13:55:46.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32ef75700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3281565e0 con 0x7fb328139330 2020-11-20T13:55:46.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb32813c560 0x7fb328159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb32813c560 0x7fb328159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39960/0 (socket says 172.21.15.117:39960) 2020-11-20T13:55:46.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 -- 172.21.15.117:0/2243395876 learned_addr learned my addr 172.21.15.117:0/2243395876 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:46.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32cd11700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb328142d30 0x7fb328156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32d512700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb328139330 0x7fb3281397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 -- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb328139330 msgr2=0x7fb3281397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:46.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb328139330 0x7fb3281397c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 -- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb328142d30 msgr2=0x7fb328156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:46.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb328142d30 0x7fb328156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:46.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 -- 172.21.15.117:0/2243395876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32815a3e0 con 0x7fb32813c560 2020-11-20T13:55:46.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32d512700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb328139330 0x7fb3281397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:46.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb327fff700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb32813c560 0x7fb328159ef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb31c00d000 tx=0x7fb31c007740).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:46.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb325ffb700 1 -- 172.21.15.117:0/2243395876 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb31c00f950 con 0x7fb32813c560 2020-11-20T13:55:46.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb325ffb700 1 -- 172.21.15.117:0/2243395876 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c00fab0 con 0x7fb32813c560 2020-11-20T13:55:46.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb328143110 con 0x7fb32813c560 2020-11-20T13:55:46.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.443+0000 7fb325ffb700 1 -- 172.21.15.117:0/2243395876 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb31c00fe10 con 0x7fb32813c560 2020-11-20T13:55:46.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.444+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3280664c0 con 0x7fb32813c560 2020-11-20T13:55:46.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.444+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3281567a0 con 0x7fb32813c560 2020-11-20T13:55:46.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.444+0000 7fb325ffb700 1 -- 172.21.15.117:0/2243395876 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb31c012350 con 0x7fb32813c560 2020-11-20T13:55:46.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.445+0000 7fb325ffb700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb308051210 0x7fb3080536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:46.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.445+0000 7fb325ffb700 1 -- 172.21.15.117:0/2243395876 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fb31c0686c0 con 0x7fb32813c560 2020-11-20T13:55:46.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.445+0000 7fb32cd11700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb308051210 0x7fb3080536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:46.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.445+0000 7fb32cd11700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb308051210 0x7fb3080536d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb318007000 tx=0x7fb31801a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:46.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.448+0000 7fb325ffb700 1 -- 172.21.15.117:0/2243395876 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb31c040b10 con 0x7fb32813c560 2020-11-20T13:55:46.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:46.609+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb328057050 con 0x7fb32813c560 2020-11-20T13:55:47.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.309+0000 7fb325ffb700 1 -- 172.21.15.117:0/2243395876 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 154+0+0 (secure 0 0 0) 0x7fb31c019030 con 0x7fb32813c560 2020-11-20T13:55:47.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb308051210 msgr2=0x7fb3080536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:47.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb308051210 0x7fb3080536d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb318007000 tx=0x7fb31801a000).stop 2020-11-20T13:55:47.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb32813c560 msgr2=0x7fb328159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:47.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb32813c560 0x7fb328159ef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb31c00d000 tx=0x7fb31c007740).stop 2020-11-20T13:55:47.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 shutdown_connections 2020-11-20T13:55:47.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb308051210 0x7fb3080536d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:47.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb328139330 0x7fb3281397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:47.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb328142d30 0x7fb328156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:47.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 --2- 172.21.15.117:0/2243395876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb32813c560 0x7fb328159ef0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:47.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 >> 172.21.15.117:0/2243395876 conn(0x7fb3280b9770 msgr2=0x7fb3280b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:47.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 shutdown_connections 2020-11-20T13:55:47.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.311+0000 7fb32ef75700 1 -- 172.21.15.117:0/2243395876 wait complete. 2020-11-20T13:55:47.325 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 erasure cephfs_data 2020-11-20T13:55:47.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.437+0000 7f21630fe700 1 Processor -- start 2020-11-20T13:55:47.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.437+0000 7f21630fe700 1 -- start start 2020-11-20T13:55:47.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f21630fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f215c09bd80 0x7f215c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:47.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f21630fe700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215c0999d0 con 0x7f215c09a110 2020-11-20T13:55:47.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f21630fe700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215c099c70 con 0x7f215c143eb0 2020-11-20T13:55:47.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f21630fe700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215c09c160 con 0x7f215c09bd80 2020-11-20T13:55:47.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f2160e9a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f215c09bd80 0x7f215c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:47.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f2160e9a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f215c09bd80 0x7f215c09cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39968/0 (socket says 172.21.15.117:39968) 2020-11-20T13:55:47.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f2160e9a700 1 -- 172.21.15.117:0/25198091 learned_addr learned my addr 172.21.15.117:0/25198091 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:47.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f216169b700 1 --1- 172.21.15.117:0/25198091 >> v1:172.21.15.117:6789/0 conn(0x7f215c143eb0 0x7f215c0b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59128/0 (socket says 172.21.15.117:59128) 2020-11-20T13:55:47.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f2160e9a700 1 -- 172.21.15.117:0/25198091 >> v1:172.21.15.117:6789/0 conn(0x7f215c143eb0 legacy=0x7f215c0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:47.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f2160e9a700 1 -- 172.21.15.117:0/25198091 >> v1:172.21.15.85:6789/0 conn(0x7f215c09a110 legacy=0x7f215c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:47.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f215b7fe700 1 -- 172.21.15.117:0/25198091 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567602578 0 0) 0x7f215c099c70 con 0x7f215c143eb0 2020-11-20T13:55:47.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f2160e9a700 1 -- 172.21.15.117:0/25198091 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c09a5c0 con 0x7f215c09bd80 2020-11-20T13:55:47.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f2160e9a700 1 --2- 172.21.15.117:0/25198091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09bd80 0x7f215c09cb40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f214c002f30 tx=0x7f214c006e80).ready entity=mon.1 client_cookie=6cbe0af12e8a3473 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:47.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.438+0000 7f215b7fe700 1 -- 172.21.15.117:0/25198091 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214c0097f0 con 0x7f215c09bd80 2020-11-20T13:55:47.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f215b7fe700 1 -- 172.21.15.117:0/25198091 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c009950 con 0x7f215c09bd80 2020-11-20T13:55:47.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f215b7fe700 1 -- 172.21.15.117:0/25198091 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214c009cb0 con 0x7f215c09bd80 2020-11-20T13:55:47.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f21630fe700 1 -- 172.21.15.117:0/25198091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09bd80 msgr2=0x7f215c09cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:47.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f21630fe700 1 --2- 172.21.15.117:0/25198091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09bd80 0x7f215c09cb40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f214c002f30 tx=0x7f214c006e80).stop 2020-11-20T13:55:47.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f21630fe700 1 -- 172.21.15.117:0/25198091 shutdown_connections 2020-11-20T13:55:47.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f21630fe700 1 --2- 172.21.15.117:0/25198091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09bd80 0x7f215c09cb40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:47.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f21630fe700 1 -- 172.21.15.117:0/25198091 >> 172.21.15.117:0/25198091 conn(0x7f215c0b9770 msgr2=0x7f215c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:47.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f21630fe700 1 -- 172.21.15.117:0/25198091 shutdown_connections 2020-11-20T13:55:47.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.439+0000 7f21630fe700 1 -- 172.21.15.117:0/25198091 wait complete. 2020-11-20T13:55:47.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.440+0000 7f21630fe700 1 Processor -- start 2020-11-20T13:55:47.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.440+0000 7f21630fe700 1 -- start start 2020-11-20T13:55:47.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09a110 0x7f215c136010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:47.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f215c09bd80 0x7f215c065440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:47.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f215c143eb0 0x7f215c09d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:47.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c065ae0 con 0x7f215c143eb0 2020-11-20T13:55:47.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c0661b0 con 0x7f215c09a110 2020-11-20T13:55:47.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215c09ee10 con 0x7f215c09bd80 2020-11-20T13:55:47.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09a110 0x7f215c136010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:47.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09a110 0x7f215c136010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39972/0 (socket says 172.21.15.117:39972) 2020-11-20T13:55:47.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 -- 172.21.15.117:0/1960973510 learned_addr learned my addr 172.21.15.117:0/1960973510 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:47.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f216169b700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f215c143eb0 0x7f215c09d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:47.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f215bfff700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f215c09bd80 0x7f215c065440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:47.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 -- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f215c09bd80 msgr2=0x7f215c065440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:47.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f215c09bd80 0x7f215c065440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:47.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 -- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f215c143eb0 msgr2=0x7f215c09d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:47.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f215c143eb0 0x7f215c09d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:47.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 -- 172.21.15.117:0/1960973510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c065d10 con 0x7f215c09a110 2020-11-20T13:55:47.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f216169b700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f215c143eb0 0x7f215c09d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:47.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f215bfff700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f215c09bd80 0x7f215c065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:47.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2160e9a700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09a110 0x7f215c136010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f214c002f30 tx=0x7f214c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:47.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2159ffb700 1 -- 172.21.15.117:0/1960973510 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214c011e90 con 0x7f215c09a110 2020-11-20T13:55:47.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2159ffb700 1 -- 172.21.15.117:0/1960973510 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c006030 con 0x7f215c09a110 2020-11-20T13:55:47.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c09f0b0 con 0x7f215c09a110 2020-11-20T13:55:47.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f2159ffb700 1 -- 172.21.15.117:0/1960973510 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f214c006390 con 0x7f215c09a110 2020-11-20T13:55:47.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.441+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c09db90 con 0x7f215c09a110 2020-11-20T13:55:47.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.443+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c065930 con 0x7f215c09a110 2020-11-20T13:55:47.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.443+0000 7f2159ffb700 1 -- 172.21.15.117:0/1960973510 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f214c01a040 con 0x7f215c09a110 2020-11-20T13:55:47.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.443+0000 7f2159ffb700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f213c051210 0x7f213c0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:47.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.443+0000 7f2159ffb700 1 -- 172.21.15.117:0/1960973510 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5863+0+0 (secure 0 0 0) 0x7f214c069e20 con 0x7f215c09a110 2020-11-20T13:55:47.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.443+0000 7f215bfff700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f213c051210 0x7f213c0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:47.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.443+0000 7f215bfff700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f213c051210 0x7f213c0536d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f215000b000 tx=0x7f2150009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:47.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.446+0000 7f2159ffb700 1 -- 172.21.15.117:0/1960973510 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f214c042320 con 0x7f215c09a110 2020-11-20T13:55:47.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:47.601+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"} v 0) v1 -- 0x7f215c057050 con 0x7f215c09a110 2020-11-20T13:55:49.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.321+0000 7f2159ffb700 1 -- 172.21.15.117:0/1960973510 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8, "pool_type": "erasure", "erasure_code_profile": "cephfs_data"}]=0 pool 'cephfs_data' created v22) v1 ==== 188+0+0 (secure 0 0 0) 0x7f214c024030 con 0x7f215c09a110 2020-11-20T13:55:49.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.325+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f213c051210 msgr2=0x7f213c0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.325+0000 7f21630fe700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f213c051210 0x7f213c0536d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f215000b000 tx=0x7f2150009040).stop 2020-11-20T13:55:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.325+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09a110 msgr2=0x7f215c136010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:49.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.325+0000 7f21630fe700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09a110 0x7f215c136010 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f214c002f30 tx=0x7f214c015000).stop 2020-11-20T13:55:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.326+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 shutdown_connections 2020-11-20T13:55:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.326+0000 7f21630fe700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f213c051210 0x7f213c0536d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.326+0000 7f21630fe700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f215c143eb0 0x7f215c09d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.326+0000 7f21630fe700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c09a110 0x7f215c136010 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0x7f214c002f30 tx=0x7f214c015000).stop 2020-11-20T13:55:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.326+0000 7f21630fe700 1 --2- 172.21.15.117:0/1960973510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f215c09bd80 0x7f215c065440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.326+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 >> 172.21.15.117:0/1960973510 conn(0x7f215c0b9770 msgr2=0x7f215c138af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.327+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 shutdown_connections 2020-11-20T13:55:49.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.327+0000 7f21630fe700 1 -- 172.21.15.117:0/1960973510 wait complete. 2020-11-20T13:55:49.329 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' created 2020-11-20T13:55:49.339 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool set cephfs_data allow_ec_overwrites true 2020-11-20T13:55:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.454+0000 7f4b49eed700 1 Processor -- start 2020-11-20T13:55:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.454+0000 7f4b49eed700 1 -- start start 2020-11-20T13:55:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.455+0000 7f4b49eed700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 0x7f4b4409b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.455+0000 7f4b49eed700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b440b4860 con 0x7f4b440b56a0 2020-11-20T13:55:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.455+0000 7f4b49eed700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b440b4b00 con 0x7f4b440b5240 2020-11-20T13:55:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.455+0000 7f4b49eed700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441439f0 con 0x7f4b44098d60 2020-11-20T13:55:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.455+0000 7f4b42ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4b440b5240 0x7f4b440630c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59140/0 (socket says 172.21.15.117:59140) 2020-11-20T13:55:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.455+0000 7f4b42ffd700 1 -- 172.21.15.117:0/1106757484 learned_addr learned my addr 172.21.15.117:0/1106757484 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.455+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399020638 0 0) 0x7f4b440b4b00 con 0x7f4b440b5240 2020-11-20T13:55:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 0x7f4b4409b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b30003230 con 0x7f4b440b5240 2020-11-20T13:55:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1354480667 0 0) 0x7f4b30003230 con 0x7f4b440b5240 2020-11-20T13:55:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 msgr2=0x7f4b4409b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 0x7f4b4409b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 >> v1:172.21.15.85:6789/0 conn(0x7f4b440b56a0 legacy=0x7f4b4409c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440b4da0 con 0x7f4b440b5240 2020-11-20T13:55:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f4b380016d0 con 0x7f4b440b5240 2020-11-20T13:55:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 >> v1:172.21.15.117:6789/0 conn(0x7f4b440b5240 legacy=0x7f4b440630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b30003720 0x7f4b30003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 0x7f4b300044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b300049c0 0x7f4b30004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441439f0 con 0x7f4b300049c0 2020-11-20T13:55:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b30004130 con 0x7f4b30003720 2020-11-20T13:55:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300042f0 con 0x7f4b44098d60 2020-11-20T13:55:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b30003720 0x7f4b30003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b42ffd700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 0x7f4b300044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b43fff700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b300049c0 0x7f4b30004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 -- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 msgr2=0x7f4b300044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44098d60 0x7f4b300044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 -- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b300049c0 msgr2=0x7f4b30004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b300049c0 0x7f4b30004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 -- 172.21.15.117:0/1106757484 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b30003230 con 0x7f4b30003720 2020-11-20T13:55:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b43fff700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b300049c0 0x7f4b30004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b30003720 0x7f4b30003b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4b3400c000 tx=0x7f4b34008040).ready entity=mon.1 client_cookie=adb51de864ab5e14 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.456+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3400eb60 con 0x7f4b30003720 2020-11-20T13:55:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3400ecc0 con 0x7f4b30003720 2020-11-20T13:55:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b427fc700 1 -- 172.21.15.117:0/1106757484 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3401a660 con 0x7f4b30003720 2020-11-20T13:55:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b49eed700 1 -- 172.21.15.117:0/1106757484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b30003720 msgr2=0x7f4b30003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b49eed700 1 --2- 172.21.15.117:0/1106757484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b30003720 0x7f4b30003b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f4b3400c000 tx=0x7f4b34008040).stop 2020-11-20T13:55:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b43fff700 1 -- 172.21.15.117:0/1106757484 reap_dead start 2020-11-20T13:55:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b49eed700 1 -- 172.21.15.117:0/1106757484 shutdown_connections 2020-11-20T13:55:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b49eed700 1 -- 172.21.15.117:0/1106757484 >> 172.21.15.117:0/1106757484 conn(0x7f4b440b9770 msgr2=0x7f4b440b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:49.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b49eed700 1 -- 172.21.15.117:0/1106757484 shutdown_connections 2020-11-20T13:55:49.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.457+0000 7f4b49eed700 1 -- 172.21.15.117:0/1106757484 wait complete. 2020-11-20T13:55:49.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.458+0000 7f4b49eed700 1 Processor -- start 2020-11-20T13:55:49.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.458+0000 7f4b49eed700 1 -- start start 2020-11-20T13:55:49.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44062f20 0x7f4b44066070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b44151c40 0x7f4b440668f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b441520c0 0x7f4b44066de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b44152550 con 0x7f4b44151c40 2020-11-20T13:55:49.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441527f0 con 0x7f4b441520c0 2020-11-20T13:55:49.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b44063300 con 0x7f4b44062f20 2020-11-20T13:55:49.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b441520c0 0x7f4b44066de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b441520c0 0x7f4b44066de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39994/0 (socket says 172.21.15.117:39994) 2020-11-20T13:55:49.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 -- 172.21.15.117:0/1575246069 learned_addr learned my addr 172.21.15.117:0/1575246069 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:49.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44062f20 0x7f4b44066070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b42ffd700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b44151c40 0x7f4b440668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 -- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44062f20 msgr2=0x7f4b44066070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:49.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44062f20 0x7f4b44066070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 -- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b44151c40 msgr2=0x7f4b440668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:49.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b44151c40 0x7f4b440668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:49.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 -- 172.21.15.117:0/1575246069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440640e0 con 0x7f4b441520c0 2020-11-20T13:55:49.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b42ffd700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b44151c40 0x7f4b440668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:49.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b43fff700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b441520c0 0x7f4b44066de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4b3c0011d0 tx=0x7f4b3c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:49.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b40ff9700 1 -- 172.21.15.117:0/1575246069 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3c010680 con 0x7f4b441520c0 2020-11-20T13:55:49.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b40ff9700 1 -- 172.21.15.117:0/1575246069 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c010040 con 0x7f4b441520c0 2020-11-20T13:55:49.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b40ff9700 1 -- 172.21.15.117:0/1575246069 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3c0109d0 con 0x7f4b441520c0 2020-11-20T13:55:49.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4409e990 con 0x7f4b441520c0 2020-11-20T13:55:49.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.459+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4409ee60 con 0x7f4b441520c0 2020-11-20T13:55:49.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.460+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4409e680 con 0x7f4b441520c0 2020-11-20T13:55:49.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.460+0000 7f4b40ff9700 1 -- 172.21.15.117:0/1575246069 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4b3c010b30 con 0x7f4b441520c0 2020-11-20T13:55:49.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.461+0000 7f4b40ff9700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4b24051210 0x7f4b240536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:49.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.461+0000 7f4b40ff9700 1 -- 172.21.15.117:0/1575246069 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f4b3c0691e0 con 0x7f4b441520c0 2020-11-20T13:55:49.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.461+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4b24051210 0x7f4b240536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:49.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.461+0000 7f4b437fe700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4b24051210 0x7f4b240536d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4b3401c000 tx=0x7f4b3401a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:49.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.464+0000 7f4b40ff9700 1 -- 172.21.15.117:0/1575246069 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f4b3c041470 con 0x7f4b441520c0 2020-11-20T13:55:49.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:49.619+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f4b44057050 con 0x7f4b441520c0 2020-11-20T13:55:50.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.324+0000 7f4b40ff9700 1 -- 172.21.15.117:0/1575246069 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 166+0+0 (secure 0 0 0) 0x7f4b3c044a90 con 0x7f4b441520c0 2020-11-20T13:55:50.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.329+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4b24051210 msgr2=0x7f4b240536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.329+0000 7f4b49eed700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4b24051210 0x7f4b240536d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4b3401c000 tx=0x7f4b3401a000).stop 2020-11-20T13:55:50.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.329+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b441520c0 msgr2=0x7f4b44066de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.329+0000 7f4b49eed700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b441520c0 0x7f4b44066de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4b3c0011d0 tx=0x7f4b3c009040).stop 2020-11-20T13:55:50.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.330+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 shutdown_connections 2020-11-20T13:55:50.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.330+0000 7f4b49eed700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4b24051210 0x7f4b240536d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.330+0000 7f4b49eed700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4b44062f20 0x7f4b44066070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.330+0000 7f4b49eed700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b441520c0 0x7f4b44066de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.330+0000 7f4b49eed700 1 --2- 172.21.15.117:0/1575246069 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4b44151c40 0x7f4b440668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.330+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 >> 172.21.15.117:0/1575246069 conn(0x7f4b440b9770 msgr2=0x7f4b440b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:50.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.331+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 shutdown_connections 2020-11-20T13:55:50.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.331+0000 7f4b49eed700 1 -- 172.21.15.117:0/1575246069 wait complete. 2020-11-20T13:55:50.334 INFO:teuthology.orchestra.run.smithi117.stderr:set pool 4 allow_ec_overwrites to true 2020-11-20T13:55:50.344 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T13:55:50.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.455+0000 7f375aa70700 1 Processor -- start 2020-11-20T13:55:50.456 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.455+0000 7f375aa70700 1 -- start start 2020-11-20T13:55:50.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375aa70700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 0x7f3754062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375aa70700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f3754147030 0x7f375409c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375aa70700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375409f100 con 0x7f375409bf70 2020-11-20T13:55:50.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375aa70700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375409ef20 con 0x7f3754147030 2020-11-20T13:55:50.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375aa70700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375409f3a0 con 0x7f375405f300 2020-11-20T13:55:50.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375926d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f375409bf70 0x7f3754146920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59160/0 (socket says 172.21.15.117:59160) 2020-11-20T13:55:50.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375926d700 1 -- 172.21.15.117:0/3481513176 learned_addr learned my addr 172.21.15.117:0/3481513176 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:50.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f375a26f700 1 --2- 172.21.15.117:0/3481513176 >> v2:172.21.15.85:3300/0 conn(0x7f3754147030 0x7f375409c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3759a6e700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 0x7f3754062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783569653 0 0) 0x7f375409f100 con 0x7f375409bf70 2020-11-20T13:55:50.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f373c003230 con 0x7f375409bf70 2020-11-20T13:55:50.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3030044272 0 0) 0x7f373c003230 con 0x7f375409bf70 2020-11-20T13:55:50.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 msgr2=0x7f3754062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:50.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 0x7f3754062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 >> v2:172.21.15.85:3300/0 conn(0x7f3754147030 msgr2=0x7f375409c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:50.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 --2- 172.21.15.117:0/3481513176 >> v2:172.21.15.85:3300/0 conn(0x7f3754147030 0x7f375409c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.456+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375405f6e0 con 0x7f375409bf70 2020-11-20T13:55:50.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f37500016d0 con 0x7f375409bf70 2020-11-20T13:55:50.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 >> v1:172.21.15.117:6789/0 conn(0x7f375409bf70 legacy=0x7f3754146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373c003720 0x7f373c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 0x7f373c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f373c0049c0 0x7f373c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375409ef20 con 0x7f373c0049c0 2020-11-20T13:55:50.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375409f3a0 con 0x7f373c003720 2020-11-20T13:55:50.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0041c0 con 0x7f375405f300 2020-11-20T13:55:50.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3759a6e700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373c003720 0x7f373c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f375a26f700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f373c0049c0 0x7f373c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f375926d700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 0x7f373c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3759a6e700 1 -- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 msgr2=0x7f373c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:50.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3759a6e700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 0x7f373c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3759a6e700 1 -- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f373c0049c0 msgr2=0x7f373c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:50.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3759a6e700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f373c0049c0 0x7f373c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3759a6e700 1 -- 172.21.15.117:0/3481513176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c003230 con 0x7f373c003720 2020-11-20T13:55:50.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f375926d700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f375405f300 0x7f373c0044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:50.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3759a6e700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373c003720 0x7f373c003b40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f374400c000 tx=0x7f3744008040).ready entity=mon.1 client_cookie=361df7cd7a7884b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:50.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374400e9d0 con 0x7f373c003720 2020-11-20T13:55:50.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400eb30 con 0x7f373c003720 2020-11-20T13:55:50.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f3758a6c700 1 -- 172.21.15.117:0/3481513176 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374400ee90 con 0x7f373c003720 2020-11-20T13:55:50.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f375aa70700 1 -- 172.21.15.117:0/3481513176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373c003720 msgr2=0x7f373c003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f375aa70700 1 --2- 172.21.15.117:0/3481513176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373c003720 0x7f373c003b40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f374400c000 tx=0x7f3744008040).stop 2020-11-20T13:55:50.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.457+0000 7f375a26f700 1 -- 172.21.15.117:0/3481513176 reap_dead start 2020-11-20T13:55:50.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.458+0000 7f375aa70700 1 -- 172.21.15.117:0/3481513176 shutdown_connections 2020-11-20T13:55:50.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.458+0000 7f375aa70700 1 -- 172.21.15.117:0/3481513176 >> 172.21.15.117:0/3481513176 conn(0x7f37540b9490 msgr2=0x7f37540b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:50.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.458+0000 7f375aa70700 1 -- 172.21.15.117:0/3481513176 shutdown_connections 2020-11-20T13:55:50.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.458+0000 7f375aa70700 1 -- 172.21.15.117:0/3481513176 wait complete. 2020-11-20T13:55:50.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 Processor -- start 2020-11-20T13:55:50.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 -- start start 2020-11-20T13:55:50.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754137710 0x7f375413c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37541445b0 0x7f37541368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3754136da0 0x7f3754157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3754137230 con 0x7f3754136da0 2020-11-20T13:55:50.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375413ff30 con 0x7f3754137710 2020-11-20T13:55:50.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375aa70700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375413c770 con 0x7f37541445b0 2020-11-20T13:55:50.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f3759a6e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754137710 0x7f375413c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f3759a6e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754137710 0x7f375413c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40010/0 (socket says 172.21.15.117:40010) 2020-11-20T13:55:50.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f3759a6e700 1 -- 172.21.15.117:0/1741431985 learned_addr learned my addr 172.21.15.117:0/1741431985 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:50.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375926d700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37541445b0 0x7f37541368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.459+0000 7f375a26f700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3754136da0 0x7f3754157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3759a6e700 1 -- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37541445b0 msgr2=0x7f37541368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3759a6e700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37541445b0 0x7f37541368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3759a6e700 1 -- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3754136da0 msgr2=0x7f3754157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3759a6e700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3754136da0 0x7f3754157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3759a6e700 1 -- 172.21.15.117:0/1741431985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3754145bc0 con 0x7f3754137710 2020-11-20T13:55:50.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3759a6e700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754137710 0x7f375413c120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f374400c000 tx=0x7f374401a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:50.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3742ffd700 1 -- 172.21.15.117:0/1741431985 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374401c620 con 0x7f3754137710 2020-11-20T13:55:50.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3742ffd700 1 -- 172.21.15.117:0/1741431985 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374401c780 con 0x7f3754137710 2020-11-20T13:55:50.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3754145e60 con 0x7f3754137710 2020-11-20T13:55:50.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f3742ffd700 1 -- 172.21.15.117:0/1741431985 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f374401cae0 con 0x7f3754137710 2020-11-20T13:55:50.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.460+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3754146300 con 0x7f3754137710 2020-11-20T13:55:50.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.461+0000 7f3742ffd700 1 -- 172.21.15.117:0/1741431985 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f374401cc40 con 0x7f3754137710 2020-11-20T13:55:50.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.461+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754145ff0 con 0x7f3754137710 2020-11-20T13:55:50.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.461+0000 7f3742ffd700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f37340513c0 0x7f3734053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.461+0000 7f3742ffd700 1 -- 172.21.15.117:0/1741431985 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f374406af30 con 0x7f3754137710 2020-11-20T13:55:50.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.461+0000 7f375926d700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f37340513c0 0x7f3734053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.462+0000 7f375926d700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f37340513c0 0x7f3734053880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f375000d000 tx=0x7f375000c040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:50.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.465+0000 7f3742ffd700 1 -- 172.21.15.117:0/1741431985 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f37440431c0 con 0x7f3754137710 2020-11-20T13:55:50.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.674+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f3754057050 con 0x7f3754137710 2020-11-20T13:55:50.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.680+0000 7f3742ffd700 1 -- 172.21.15.117:0/1741431985 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7f3744016070 con 0x7f3754137710 2020-11-20T13:55:50.683 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f37340513c0 msgr2=0x7f3734053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.683 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f37340513c0 0x7f3734053880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f375000d000 tx=0x7f375000c040).stop 2020-11-20T13:55:50.684 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754137710 msgr2=0x7f375413c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.684 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754137710 0x7f375413c120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f374400c000 tx=0x7f374401a000).stop 2020-11-20T13:55:50.686 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 shutdown_connections 2020-11-20T13:55:50.686 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f37340513c0 0x7f3734053880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.686 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3754136da0 0x7f3754157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.687 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3754137710 0x7f375413c120 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.687 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 --2- 172.21.15.117:0/1741431985 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37541445b0 0x7f37541368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.688 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.683+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 >> 172.21.15.117:0/1741431985 conn(0x7f37540b9490 msgr2=0x7f37540635d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:50.688 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.684+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 shutdown_connections 2020-11-20T13:55:50.688 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.684+0000 7f375aa70700 1 -- 172.21.15.117:0/1741431985 wait complete. 2020-11-20T13:55:50.691 INFO:teuthology.orchestra.run.smithi117.stderr:new fs with metadata pool 3 and data pool 4 2020-11-20T13:55:50.704 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:55:50.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.810+0000 7f10beb99700 1 Processor -- start 2020-11-20T13:55:50.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.810+0000 7f10beb99700 1 -- start start 2020-11-20T13:55:50.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10beb99700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f10b8143d40 0x7f10b80aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10beb99700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b809b920 con 0x7f10b81438e0 2020-11-20T13:55:50.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10beb99700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b81441b0 con 0x7f10b8098e30 2020-11-20T13:55:50.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10beb99700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b80b5510 con 0x7f10b8143d40 2020-11-20T13:55:50.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10bc935700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f10b8098e30 0x7f10b809b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59188/0 (socket says 172.21.15.117:59188) 2020-11-20T13:55:50.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10bc935700 1 -- 172.21.15.117:0/3656565644 learned_addr learned my addr 172.21.15.117:0/3656565644 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:50.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10bd136700 1 --2- 172.21.15.117:0/3656565644 >> v2:172.21.15.117:3300/0 conn(0x7f10b8143d40 0x7f10b80aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10bd136700 1 -- 172.21.15.117:0/3656565644 >> v1:172.21.15.117:6789/0 conn(0x7f10b8098e30 legacy=0x7f10b809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10bd136700 1 -- 172.21.15.117:0/3656565644 >> v1:172.21.15.85:6789/0 conn(0x7f10b81438e0 legacy=0x7f10b80ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:50.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10b77fe700 1 -- 172.21.15.117:0/3656565644 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667885989 0 0) 0x7f10b81441b0 con 0x7f10b8098e30 2020-11-20T13:55:50.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.811+0000 7f10bd136700 1 -- 172.21.15.117:0/3656565644 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80b57b0 con 0x7f10b8143d40 2020-11-20T13:55:50.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10bd136700 1 --2- 172.21.15.117:0/3656565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b8143d40 0x7f10b80aee00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f10b80b5dd0 tx=0x7f10b00091a0).ready entity=mon.1 client_cookie=107a5d356111a485 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:50.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10b77fe700 1 -- 172.21.15.117:0/3656565644 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10b000ba10 con 0x7f10b8143d40 2020-11-20T13:55:50.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10b77fe700 1 -- 172.21.15.117:0/3656565644 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b000bb70 con 0x7f10b8143d40 2020-11-20T13:55:50.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10b77fe700 1 -- 172.21.15.117:0/3656565644 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10b000bce0 con 0x7f10b8143d40 2020-11-20T13:55:50.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10beb99700 1 -- 172.21.15.117:0/3656565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b8143d40 msgr2=0x7f10b80aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10beb99700 1 --2- 172.21.15.117:0/3656565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b8143d40 0x7f10b80aee00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f10b80b5dd0 tx=0x7f10b00091a0).stop 2020-11-20T13:55:50.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10beb99700 1 -- 172.21.15.117:0/3656565644 shutdown_connections 2020-11-20T13:55:50.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10beb99700 1 --2- 172.21.15.117:0/3656565644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b8143d40 0x7f10b80aee00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10beb99700 1 -- 172.21.15.117:0/3656565644 >> 172.21.15.117:0/3656565644 conn(0x7f10b805b9f0 msgr2=0x7f10b805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:50.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10beb99700 1 -- 172.21.15.117:0/3656565644 shutdown_connections 2020-11-20T13:55:50.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.812+0000 7f10beb99700 1 -- 172.21.15.117:0/3656565644 wait complete. 2020-11-20T13:55:50.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.813+0000 7f10beb99700 1 Processor -- start 2020-11-20T13:55:50.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.813+0000 7f10beb99700 1 -- start start 2020-11-20T13:55:50.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10beb99700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10b8098e30 0x7f10b81379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10beb99700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b81438e0 0x7f10b81446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10beb99700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10b8143d40 0x7f10b8144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10beb99700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8143200 con 0x7f10b8098e30 2020-11-20T13:55:50.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10beb99700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80b2d80 con 0x7f10b81438e0 2020-11-20T13:55:50.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10beb99700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80b3020 con 0x7f10b8143d40 2020-11-20T13:55:50.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b81438e0 0x7f10b81446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b81438e0 0x7f10b81446f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40038/0 (socket says 172.21.15.117:40038) 2020-11-20T13:55:50.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 -- 172.21.15.117:0/3209726345 learned_addr learned my addr 172.21.15.117:0/3209726345 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:50.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10bc935700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10b8098e30 0x7f10b81379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10bd136700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10b8143d40 0x7f10b8144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 -- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10b8143d40 msgr2=0x7f10b8144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10b8143d40 0x7f10b8144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 -- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10b8098e30 msgr2=0x7f10b81379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10b8098e30 0x7f10b81379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10b7fff700 1 -- 172.21.15.117:0/3209726345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8142e80 con 0x7f10b81438e0 2020-11-20T13:55:50.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.814+0000 7f10bd136700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10b8143d40 0x7f10b8144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:50.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.815+0000 7f10b7fff700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b81438e0 0x7f10b81446f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f10b8146b70 tx=0x7f10ac006db0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:50.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.815+0000 7f10b5ffb700 1 -- 172.21.15.117:0/3209726345 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10ac00d4a0 con 0x7f10b81438e0 2020-11-20T13:55:50.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.815+0000 7f10b5ffb700 1 -- 172.21.15.117:0/3209726345 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac00d600 con 0x7f10b81438e0 2020-11-20T13:55:50.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.815+0000 7f10b5ffb700 1 -- 172.21.15.117:0/3209726345 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10ac00d960 con 0x7f10b81438e0 2020-11-20T13:55:50.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.815+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b8145f60 con 0x7f10b81438e0 2020-11-20T13:55:50.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.815+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b8145480 con 0x7f10b81438e0 2020-11-20T13:55:50.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.816+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b80b3220 con 0x7f10b81438e0 2020-11-20T13:55:50.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.816+0000 7f10b5ffb700 1 -- 172.21.15.117:0/3209726345 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f10ac00f070 con 0x7f10b81438e0 2020-11-20T13:55:50.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.816+0000 7f10b5ffb700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1098051210 0x7f10980536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:50.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.816+0000 7f10b5ffb700 1 -- 172.21.15.117:0/3209726345 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f10ac0689d0 con 0x7f10b81438e0 2020-11-20T13:55:50.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.816+0000 7f10bc935700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1098051210 0x7f10980536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:50.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.816+0000 7f10bc935700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1098051210 0x7f10980536d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f10a800b000 tx=0x7f10a8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:50.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.819+0000 7f10b5ffb700 1 -- 172.21.15.117:0/3209726345 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f10ac040c00 con 0x7f10b81438e0 2020-11-20T13:55:50.970 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.970+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10b8057050 con 0x7f10b81438e0 2020-11-20T13:55:50.971 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.971+0000 7f10b5ffb700 1 -- 172.21.15.117:0/3209726345 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16360 (secure 0 0 0) 0x7f10ac044220 con 0x7f10b81438e0 2020-11-20T13:55:50.971 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:50.972 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":24,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:50.675051+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:50.974 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1098051210 msgr2=0x7f10980536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.974 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1098051210 0x7f10980536d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f10a800b000 tx=0x7f10a8009040).stop 2020-11-20T13:55:50.974 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b81438e0 msgr2=0x7f10b81446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:50.975 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b81438e0 0x7f10b81446f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f10b8146b70 tx=0x7f10ac006db0).stop 2020-11-20T13:55:50.975 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 shutdown_connections 2020-11-20T13:55:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1098051210 0x7f10980536d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10b8143d40 0x7f10b8144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10b8098e30 0x7f10b81379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 --2- 172.21.15.117:0/3209726345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10b81438e0 0x7f10b81446f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:50.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 >> 172.21.15.117:0/3209726345 conn(0x7f10b805b9f0 msgr2=0x7f10b80b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:50.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 shutdown_connections 2020-11-20T13:55:50.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:50.972+0000 7f10beb99700 1 -- 172.21.15.117:0/3209726345 wait complete. 2020-11-20T13:55:50.983 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:55:51.088 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.088+0000 7fc7ecdba700 1 Processor -- start 2020-11-20T13:55:51.088 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.088+0000 7fc7ecdba700 1 -- start start 2020-11-20T13:55:51.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7ecdba700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7e81438e0 0x7fc7e8143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7ecdba700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e809b920 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7ecdba700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e80b5510 con 0x7fc7e8098e30 2020-11-20T13:55:51.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7ecdba700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e8144230 con 0x7fc7e81438e0 2020-11-20T13:55:51.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e5d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7e81438e0 0x7fc7e8143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e5d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7e81438e0 0x7fc7e8143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40044/0 (socket says 172.21.15.117:40044) 2020-11-20T13:55:51.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e659c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc7e8098e30 0x7fc7e809b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59202/0 (socket says 172.21.15.117:59202) 2020-11-20T13:55:51.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/2930775570 learned_addr learned my addr 172.21.15.117:0/2930775570 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:51.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/2930775570 >> v1:172.21.15.117:6789/0 conn(0x7fc7e8098e30 legacy=0x7fc7e809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/2930775570 >> v1:172.21.15.85:6789/0 conn(0x7fc7e80ae6f0 legacy=0x7fc7e80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:51.093 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e559a700 1 -- 172.21.15.117:0/2930775570 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007315747 0 0) 0x7fc7e80b5510 con 0x7fc7e8098e30 2020-11-20T13:55:51.093 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/2930775570 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e80b57b0 con 0x7fc7e81438e0 2020-11-20T13:55:51.093 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.089+0000 7fc7e5d9b700 1 --2- 172.21.15.117:0/2930775570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e81438e0 0x7fc7e8143d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fc7d8002f30 tx=0x7fc7d8006e80).ready entity=mon.1 client_cookie=75b3df61114ef604 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.093 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7e559a700 1 -- 172.21.15.117:0/2930775570 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d80097f0 con 0x7fc7e81438e0 2020-11-20T13:55:51.094 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7e559a700 1 -- 172.21.15.117:0/2930775570 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d8009950 con 0x7fc7e81438e0 2020-11-20T13:55:51.094 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7e559a700 1 -- 172.21.15.117:0/2930775570 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d8009cb0 con 0x7fc7e81438e0 2020-11-20T13:55:51.094 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7ecdba700 1 -- 172.21.15.117:0/2930775570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e81438e0 msgr2=0x7fc7e8143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.094 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/2930775570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e81438e0 0x7fc7e8143d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fc7d8002f30 tx=0x7fc7d8006e80).stop 2020-11-20T13:55:51.094 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7ecdba700 1 -- 172.21.15.117:0/2930775570 shutdown_connections 2020-11-20T13:55:51.094 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/2930775570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e81438e0 0x7fc7e8143d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.095 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7ecdba700 1 -- 172.21.15.117:0/2930775570 >> 172.21.15.117:0/2930775570 conn(0x7fc7e805b9f0 msgr2=0x7fc7e805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:51.095 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7ecdba700 1 -- 172.21.15.117:0/2930775570 shutdown_connections 2020-11-20T13:55:51.095 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.090+0000 7fc7ecdba700 1 -- 172.21.15.117:0/2930775570 wait complete. 2020-11-20T13:55:51.098 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.091+0000 7fc7ecdba700 1 Processor -- start 2020-11-20T13:55:51.098 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.091+0000 7fc7ecdba700 1 -- start start 2020-11-20T13:55:51.098 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc7e8098e30 0x7fc7e81491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.098 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e80ae6f0 0x7fc7e8153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.099 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc7e81438e0 0x7fc7e809e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.099 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81496a0 con 0x7fc7e81438e0 2020-11-20T13:55:51.099 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81498c0 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.099 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8149b60 con 0x7fc7e8098e30 2020-11-20T13:55:51.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e80ae6f0 0x7fc7e8153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e80ae6f0 0x7fc7e8153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40050/0 (socket says 172.21.15.117:40050) 2020-11-20T13:55:51.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/3402400500 learned_addr learned my addr 172.21.15.117:0/3402400500 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:51.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/3402400500 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc7e8098e30 msgr2=0x7fc7e81491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc7e8098e30 0x7fc7e81491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:51.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/3402400500 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc7e81438e0 msgr2=0x7fc7e809e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e6d9d700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc7e81438e0 0x7fc7e809e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc7e81438e0 0x7fc7e809e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 -- 172.21.15.117:0/3402400500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e8149e00 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7e5d9b700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e80ae6f0 0x7fc7e8153d30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc7e8148800 tx=0x7fc7d8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7d77fe700 1 -- 172.21.15.117:0/3402400500 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d8009c50 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7d77fe700 1 -- 172.21.15.117:0/3402400500 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d801e460 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e814a2a0 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7d77fe700 1 -- 172.21.15.117:0/3402400500 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7d801e7c0 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.092+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e809eca0 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.094+0000 7fc7d77fe700 1 -- 172.21.15.117:0/3402400500 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc7d8025030 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.094+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e8149f90 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.094+0000 7fc7d77fe700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc7c4051290 0x7fc7c4053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.094+0000 7fc7d77fe700 1 -- 172.21.15.117:0/3402400500 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc7d806a600 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.094+0000 7fc7e659c700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc7c4051290 0x7fc7c4053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.104 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.094+0000 7fc7e659c700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc7c4051290 0x7fc7c4053750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc7dc000b80 tx=0x7fc7dc007630).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.104 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.097+0000 7fc7d77fe700 1 -- 172.21.15.117:0/3402400500 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc7d8042740 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.246+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7e8057050 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.247+0000 7fc7d77fe700 1 -- 172.21.15.117:0/3402400500 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16360 (secure 0 0 0) 0x7fc7d8023090 con 0x7fc7e80ae6f0 2020-11-20T13:55:51.248 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:51.248 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":24,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:50.675051+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:51.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.248+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc7c4051290 msgr2=0x7fc7c4053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.248+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc7c4051290 0x7fc7c4053750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc7dc000b80 tx=0x7fc7dc007630).stop 2020-11-20T13:55:51.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.248+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e80ae6f0 msgr2=0x7fc7e8153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.248+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e80ae6f0 0x7fc7e8153d30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc7e8148800 tx=0x7fc7d8015000).stop 2020-11-20T13:55:51.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.248+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 shutdown_connections 2020-11-20T13:55:51.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.249+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc7c4051290 0x7fc7c4053750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.249+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc7e8098e30 0x7fc7e81491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.249+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc7e81438e0 0x7fc7e809e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.252 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.249+0000 7fc7ecdba700 1 --2- 172.21.15.117:0/3402400500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e80ae6f0 0x7fc7e8153d30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.253 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.249+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 >> 172.21.15.117:0/3402400500 conn(0x7fc7e805b9f0 msgr2=0x7fc7e80b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:51.253 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.249+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 shutdown_connections 2020-11-20T13:55:51.253 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.249+0000 7fc7ecdba700 1 -- 172.21.15.117:0/3402400500 wait complete. 2020-11-20T13:55:51.259 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-20T13:55:51.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.368+0000 7fe564d85700 1 Processor -- start 2020-11-20T13:55:51.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.369+0000 7fe564d85700 1 -- start start 2020-11-20T13:55:51.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.369+0000 7fe564d85700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe560098d60 0x7fe56009b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.369+0000 7fe564d85700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600b4860 con 0x7fe5600b56d0 2020-11-20T13:55:51.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.369+0000 7fe564d85700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600b4b00 con 0x7fe5600b5240 2020-11-20T13:55:51.372 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.369+0000 7fe564d85700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5601439f0 con 0x7fe560098d60 2020-11-20T13:55:51.372 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.369+0000 7fe55e59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe560098d60 0x7fe56009b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.372 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55dd9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe5600b5240 0x7fe5600630c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59216/0 (socket says 172.21.15.117:59216) 2020-11-20T13:55:51.372 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55dd9b700 1 -- 172.21.15.117:0/469742003 learned_addr learned my addr 172.21.15.117:0/469742003 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:51.372 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55e59c700 1 -- 172.21.15.117:0/469742003 >> v1:172.21.15.117:6789/0 conn(0x7fe5600b5240 legacy=0x7fe5600630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55e59c700 1 -- 172.21.15.117:0/469742003 >> v1:172.21.15.85:6789/0 conn(0x7fe5600b56d0 legacy=0x7fe56009c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55e59c700 1 -- 172.21.15.117:0/469742003 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600b4da0 con 0x7fe560098d60 2020-11-20T13:55:51.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55e59c700 1 --2- 172.21.15.117:0/469742003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe560098d60 0x7fe56009b130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fe554002f30 tx=0x7fe554006dc0).ready entity=mon.1 client_cookie=709e0a3d6cee2a0a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55d59a700 1 -- 172.21.15.117:0/469742003 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5540097f0 con 0x7fe560098d60 2020-11-20T13:55:51.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55d59a700 1 -- 172.21.15.117:0/469742003 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554009950 con 0x7fe560098d60 2020-11-20T13:55:51.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe55d59a700 1 -- 172.21.15.117:0/469742003 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe554009cb0 con 0x7fe560098d60 2020-11-20T13:55:51.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe564d85700 1 -- 172.21.15.117:0/469742003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe560098d60 msgr2=0x7fe56009b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.370+0000 7fe564d85700 1 --2- 172.21.15.117:0/469742003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe560098d60 0x7fe56009b130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fe554002f30 tx=0x7fe554006dc0).stop 2020-11-20T13:55:51.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.371+0000 7fe564d85700 1 -- 172.21.15.117:0/469742003 shutdown_connections 2020-11-20T13:55:51.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.371+0000 7fe564d85700 1 --2- 172.21.15.117:0/469742003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe560098d60 0x7fe56009b130 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.371+0000 7fe564d85700 1 -- 172.21.15.117:0/469742003 >> 172.21.15.117:0/469742003 conn(0x7fe5600b9770 msgr2=0x7fe5600b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:51.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.371+0000 7fe564d85700 1 -- 172.21.15.117:0/469742003 shutdown_connections 2020-11-20T13:55:51.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.371+0000 7fe564d85700 1 -- 172.21.15.117:0/469742003 wait complete. 2020-11-20T13:55:51.377 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 Processor -- start 2020-11-20T13:55:51.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 -- start start 2020-11-20T13:55:51.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe560098d60 0x7fe560158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5600b5240 0x7fe5600b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5600b56d0 0x7fe5600aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600b2260 con 0x7fe5600b5240 2020-11-20T13:55:51.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600b2500 con 0x7fe5600b56d0 2020-11-20T13:55:51.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe564d85700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600b27a0 con 0x7fe560098d60 2020-11-20T13:55:51.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe55ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5600b56d0 0x7fe5600aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe55ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5600b56d0 0x7fe5600aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40064/0 (socket says 172.21.15.117:40064) 2020-11-20T13:55:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe55ed9d700 1 -- 172.21.15.117:0/522397624 learned_addr learned my addr 172.21.15.117:0/522397624 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe55e59c700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe560098d60 0x7fe560158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.372+0000 7fe55dd9b700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5600b5240 0x7fe5600b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe55ed9d700 1 -- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe560098d60 msgr2=0x7fe560158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe55ed9d700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe560098d60 0x7fe560158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe55ed9d700 1 -- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5600b5240 msgr2=0x7fe5600b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe55ed9d700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5600b5240 0x7fe5600b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe55ed9d700 1 -- 172.21.15.117:0/522397624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600b3130 con 0x7fe5600b56d0 2020-11-20T13:55:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe55ed9d700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5600b56d0 0x7fe5600aed80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe56009b040 tx=0x7fe558007e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe54f7fe700 1 -- 172.21.15.117:0/522397624 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55800a5a0 con 0x7fe5600b56d0 2020-11-20T13:55:51.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe54f7fe700 1 -- 172.21.15.117:0/522397624 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800a700 con 0x7fe5600b56d0 2020-11-20T13:55:51.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5600af930 con 0x7fe5600b56d0 2020-11-20T13:55:51.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe54f7fe700 1 -- 172.21.15.117:0/522397624 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55800aa60 con 0x7fe5600b56d0 2020-11-20T13:55:51.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.373+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5601432d0 con 0x7fe5600b56d0 2020-11-20T13:55:51.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.374+0000 7fe54f7fe700 1 -- 172.21.15.117:0/522397624 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe558019030 con 0x7fe5600b56d0 2020-11-20T13:55:51.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.374+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5600b29a0 con 0x7fe5600b56d0 2020-11-20T13:55:51.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.375+0000 7fe54f7fe700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe53c0513c0 0x7fe53c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.375+0000 7fe54f7fe700 1 -- 172.21.15.117:0/522397624 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe558068910 con 0x7fe5600b56d0 2020-11-20T13:55:51.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.375+0000 7fe55e59c700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe53c0513c0 0x7fe53c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.375+0000 7fe55e59c700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe53c0513c0 0x7fe53c053880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe55401b000 tx=0x7fe554015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.384 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.378+0000 7fe54f7fe700 1 -- 172.21.15.117:0/522397624 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe5600b29a0 con 0x7fe5600b56d0 2020-11-20T13:55:51.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.580+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe5600b29a0 con 0x7fe5600b56d0 2020-11-20T13:55:51.688 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.688+0000 7fe54f7fe700 1 -- 172.21.15.117:0/522397624 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe5600b29a0 con 0x7fe5600b56d0 2020-11-20T13:55:51.690 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.689+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe53c0513c0 msgr2=0x7fe53c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.690 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.689+0000 7fe564d85700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe53c0513c0 0x7fe53c053880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe55401b000 tx=0x7fe554015000).stop 2020-11-20T13:55:51.690 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.689+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5600b56d0 msgr2=0x7fe5600aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.691 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.689+0000 7fe564d85700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5600b56d0 0x7fe5600aed80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe56009b040 tx=0x7fe558007e50).stop 2020-11-20T13:55:51.691 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 shutdown_connections 2020-11-20T13:55:51.691 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe53c0513c0 0x7fe53c053880 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.691 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe560098d60 0x7fe560158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.692 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5600b56d0 0x7fe5600aed80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.692 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 --2- 172.21.15.117:0/522397624 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe5600b5240 0x7fe5600b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.692 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 >> 172.21.15.117:0/522397624 conn(0x7fe5600b9770 msgr2=0x7fe56013b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:51.692 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 shutdown_connections 2020-11-20T13:55:51.692 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.690+0000 7fe564d85700 1 -- 172.21.15.117:0/522397624 wait complete. 2020-11-20T13:55:51.700 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:55:51.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.807+0000 7f9e6a0d3700 1 Processor -- start 2020-11-20T13:55:51.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.808+0000 7f9e6a0d3700 1 -- start start 2020-11-20T13:55:51.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.808+0000 7f9e6a0d3700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f9e640ae8b0 0x7f9e640aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.808+0000 7f9e6a0d3700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640b5120 con 0x7f9e64099fa0 2020-11-20T13:55:51.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.808+0000 7f9e6a0d3700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640b53c0 con 0x7f9e640b5860 2020-11-20T13:55:51.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.808+0000 7f9e6a0d3700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e640b4f10 con 0x7f9e640ae8b0 2020-11-20T13:55:51.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.808+0000 7f9e62ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9e640b5860 0x7f9e64098b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59232/0 (socket says 172.21.15.117:59232) 2020-11-20T13:55:51.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.808+0000 7f9e62ffd700 1 -- 172.21.15.117:0/822431691 learned_addr learned my addr 172.21.15.117:0/822431691 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:51.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 --2- 172.21.15.117:0/822431691 >> v2:172.21.15.85:3300/0 conn(0x7f9e640ae8b0 0x7f9e640aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388324279 0 0) 0x7f9e640b53c0 con 0x7f9e640b5860 2020-11-20T13:55:51.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e50003230 con 0x7f9e640b5860 2020-11-20T13:55:51.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 362366901 0 0) 0x7f9e50003230 con 0x7f9e640b5860 2020-11-20T13:55:51.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 >> v2:172.21.15.85:3300/0 conn(0x7f9e640ae8b0 msgr2=0x7f9e640aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:51.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 --2- 172.21.15.117:0/822431691 >> v2:172.21.15.85:3300/0 conn(0x7f9e640ae8b0 0x7f9e640aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 >> v1:172.21.15.85:6789/0 conn(0x7f9e64099fa0 legacy=0x7f9e6409a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640b4ac0 con 0x7f9e640b5860 2020-11-20T13:55:51.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f9e580016d0 con 0x7f9e640b5860 2020-11-20T13:55:51.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 >> v1:172.21.15.117:6789/0 conn(0x7f9e640b5860 legacy=0x7f9e64098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e50003720 0x7f9e50003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e500044d0 0x7f9e50004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e50004e40 0x7f9e5000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640b4f10 con 0x7f9e500044d0 2020-11-20T13:55:51.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50004340 con 0x7f9e50003720 2020-11-20T13:55:51.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5000bc40 con 0x7f9e50004e40 2020-11-20T13:55:51.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e50003720 0x7f9e50003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e62ffd700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e500044d0 0x7f9e50004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e63fff700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e50004e40 0x7f9e5000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 -- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e50004e40 msgr2=0x7f9e5000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:51.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e50004e40 0x7f9e5000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 -- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e500044d0 msgr2=0x7f9e50004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:51.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e500044d0 0x7f9e50004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 -- 172.21.15.117:0/822431691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50003230 con 0x7f9e50003720 2020-11-20T13:55:51.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e63fff700 1 -- 172.21.15.117:0/822431691 reap_dead start 2020-11-20T13:55:51.820 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e62ffd700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e500044d0 0x7f9e50004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:51.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e637fe700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e50003720 0x7f9e50003b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9e5400c000 tx=0x7f9e54008040).ready entity=mon.1 client_cookie=49d0ac2b1e6d17be server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5400ebc0 con 0x7f9e50003720 2020-11-20T13:55:51.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.809+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5400ed20 con 0x7f9e50003720 2020-11-20T13:55:51.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e627fc700 1 -- 172.21.15.117:0/822431691 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5401a660 con 0x7f9e50003720 2020-11-20T13:55:51.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/822431691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e50003720 msgr2=0x7f9e50003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e50003720 0x7f9e50003b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9e5400c000 tx=0x7f9e54008040).stop 2020-11-20T13:55:51.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/822431691 shutdown_connections 2020-11-20T13:55:51.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/822431691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e50003720 0x7f9e50003b40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/822431691 >> 172.21.15.117:0/822431691 conn(0x7f9e6405b9f0 msgr2=0x7f9e6405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:51.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/822431691 shutdown_connections 2020-11-20T13:55:51.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.810+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/822431691 wait complete. 2020-11-20T13:55:51.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.811+0000 7f9e6a0d3700 1 Processor -- start 2020-11-20T13:55:51.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.811+0000 7f9e6a0d3700 1 -- start start 2020-11-20T13:55:51.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e50003720 0x7f9e640912d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e64066f20 0x7f9e641562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e6414ff90 0x7f9e64150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640673a0 con 0x7f9e50003720 2020-11-20T13:55:51.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641567c0 con 0x7f9e6414ff90 2020-11-20T13:55:51.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64156a10 con 0x7f9e64066f20 2020-11-20T13:55:51.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e6414ff90 0x7f9e64150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e6414ff90 0x7f9e64150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40084/0 (socket says 172.21.15.117:40084) 2020-11-20T13:55:51.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 -- 172.21.15.117:0/704007589 learned_addr learned my addr 172.21.15.117:0/704007589 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:51.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e637fe700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e50003720 0x7f9e640912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e62ffd700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e64066f20 0x7f9e641562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 -- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e64066f20 msgr2=0x7f9e641562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e64066f20 0x7f9e641562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 -- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e50003720 msgr2=0x7f9e640912d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:51.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e50003720 0x7f9e640912d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:51.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 -- 172.21.15.117:0/704007589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64156cb0 con 0x7f9e6414ff90 2020-11-20T13:55:51.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e62ffd700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e64066f20 0x7f9e641562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:51.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e63fff700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e6414ff90 0x7f9e64150410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f9e64153ee0 tx=0x7f9e5c009040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e60ff9700 1 -- 172.21.15.117:0/704007589 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5c0032c0 con 0x7f9e6414ff90 2020-11-20T13:55:51.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e60ff9700 1 -- 172.21.15.117:0/704007589 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c010040 con 0x7f9e6414ff90 2020-11-20T13:55:51.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e60ff9700 1 -- 172.21.15.117:0/704007589 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5c001930 con 0x7f9e6414ff90 2020-11-20T13:55:51.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e64156f50 con 0x7f9e6414ff90 2020-11-20T13:55:51.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.812+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e641598a0 con 0x7f9e6414ff90 2020-11-20T13:55:51.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.813+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e641570e0 con 0x7f9e6414ff90 2020-11-20T13:55:51.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.813+0000 7f9e60ff9700 1 -- 172.21.15.117:0/704007589 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9e5c001a90 con 0x7f9e6414ff90 2020-11-20T13:55:51.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.814+0000 7f9e60ff9700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9e440513c0 0x7f9e44053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:51.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.814+0000 7f9e60ff9700 1 -- 172.21.15.117:0/704007589 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9e5c0689e0 con 0x7f9e6414ff90 2020-11-20T13:55:51.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.814+0000 7f9e637fe700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9e440513c0 0x7f9e44053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:51.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.814+0000 7f9e637fe700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9e440513c0 0x7f9e44053880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9e54007000 tx=0x7f9e5401a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:51.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:51.817+0000 7f9e60ff9700 1 -- 172.21.15.117:0/704007589 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9e5c040c00 con 0x7f9e6414ff90 2020-11-20T13:55:51.948 INFO:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:51.951 INFO:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:55:52.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.023+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e64057050 con 0x7f9e6414ff90 2020-11-20T13:55:52.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.023+0000 7f9e60ff9700 1 -- 172.21.15.117:0/704007589 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7f9e5c044220 con 0x7f9e6414ff90 2020-11-20T13:55:52.024 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:52.024 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:55:51.676934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/1587825839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1587825839},{"type":"v1","addr":"172.21.15.117:6835","nonce":1587825839}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:55:52.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9e440513c0 msgr2=0x7f9e44053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9e440513c0 0x7f9e44053880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9e54007000 tx=0x7f9e5401a000).stop 2020-11-20T13:55:52.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e6414ff90 msgr2=0x7f9e64150410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e6414ff90 0x7f9e64150410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f9e64153ee0 tx=0x7f9e5c009040).stop 2020-11-20T13:55:52.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 shutdown_connections 2020-11-20T13:55:52.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9e440513c0 0x7f9e44053880 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e6414ff90 0x7f9e64150410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9e50003720 0x7f9e640912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 --2- 172.21.15.117:0/704007589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9e64066f20 0x7f9e641562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 >> 172.21.15.117:0/704007589 conn(0x7f9e6405b9f0 msgr2=0x7f9e640b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:52.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 shutdown_connections 2020-11-20T13:55:52.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.025+0000 7f9e6a0d3700 1 -- 172.21.15.117:0/704007589 wait complete. 2020-11-20T13:55:52.028 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2020-11-20T13:55:52.035 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:55:52.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.181+0000 7efe8984d700 1 Processor -- start 2020-11-20T13:55:52.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.181+0000 7efe8984d700 1 -- start start 2020-11-20T13:55:52.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.181+0000 7efe8984d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7efe84098e30 0x7efe8409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.181+0000 7efe8984d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe8409b700 0x7efe841438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe8984d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe8984d700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe840b5510 con 0x7efe84098e30 2020-11-20T13:55:52.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe8984d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe840b57b0 con 0x7efe8409b700 2020-11-20T13:55:52.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe8984d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe840b5a50 con 0x7efe84143dd0 2020-11-20T13:55:52.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe8409b700 0x7efe841438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe8409b700 0x7efe841438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40088/0 (socket says 172.21.15.117:40088) 2020-11-20T13:55:52.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 -- 172.21.15.117:0/2854730759 learned_addr learned my addr 172.21.15.117:0/2854730759 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:52.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe82ffd700 1 --2- 172.21.15.117:0/2854730759 >> v2:172.21.15.85:3300/0 conn(0x7efe84098e30 0x7efe8409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe837fe700 1 --2- 172.21.15.117:0/2854730759 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 -- 172.21.15.117:0/2854730759 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 msgr2=0x7efe840b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:52.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 --2- 172.21.15.117:0/2854730759 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe840b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 -- 172.21.15.117:0/2854730759 >> v2:172.21.15.85:3300/0 conn(0x7efe84098e30 msgr2=0x7efe8409b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:52.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 --2- 172.21.15.117:0/2854730759 >> v2:172.21.15.85:3300/0 conn(0x7efe84098e30 0x7efe8409b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 -- 172.21.15.117:0/2854730759 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84099210 con 0x7efe8409b700 2020-11-20T13:55:52.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe827fc700 1 --2- 172.21.15.117:0/2854730759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe841438e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7efe74002f30 tx=0x7efe74006e80).ready entity=mon.1 client_cookie=a1e4be115964572a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe81ffb700 1 -- 172.21.15.117:0/2854730759 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe740097f0 con 0x7efe8409b700 2020-11-20T13:55:52.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe81ffb700 1 -- 172.21.15.117:0/2854730759 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe74009950 con 0x7efe8409b700 2020-11-20T13:55:52.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe81ffb700 1 -- 172.21.15.117:0/2854730759 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe74009cb0 con 0x7efe8409b700 2020-11-20T13:55:52.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe8984d700 1 -- 172.21.15.117:0/2854730759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 msgr2=0x7efe841438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.182+0000 7efe8984d700 1 --2- 172.21.15.117:0/2854730759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe841438e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7efe74002f30 tx=0x7efe74006e80).stop 2020-11-20T13:55:52.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.183+0000 7efe8984d700 1 -- 172.21.15.117:0/2854730759 shutdown_connections 2020-11-20T13:55:52.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.183+0000 7efe8984d700 1 --2- 172.21.15.117:0/2854730759 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe840b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.183+0000 7efe8984d700 1 --2- 172.21.15.117:0/2854730759 >> v2:172.21.15.85:3300/0 conn(0x7efe84098e30 0x7efe8409b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.183+0000 7efe8984d700 1 --2- 172.21.15.117:0/2854730759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe841438e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.183+0000 7efe8984d700 1 -- 172.21.15.117:0/2854730759 >> 172.21.15.117:0/2854730759 conn(0x7efe8405b9f0 msgr2=0x7efe8405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:52.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.183+0000 7efe8984d700 1 -- 172.21.15.117:0/2854730759 shutdown_connections 2020-11-20T13:55:52.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.183+0000 7efe8984d700 1 -- 172.21.15.117:0/2854730759 wait complete. 2020-11-20T13:55:52.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 Processor -- start 2020-11-20T13:55:52.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 -- start start 2020-11-20T13:55:52.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe84098e30 0x7efe841459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe840b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe84144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe84157030 con 0x7efe84098e30 2020-11-20T13:55:52.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe841572d0 con 0x7efe8409b700 2020-11-20T13:55:52.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.184+0000 7efe8984d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe840b1ec0 con 0x7efe84143dd0 2020-11-20T13:55:52.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe840b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe840b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40096/0 (socket says 172.21.15.117:40096) 2020-11-20T13:55:52.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 -- 172.21.15.117:0/764058031 learned_addr learned my addr 172.21.15.117:0/764058031 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:52.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe82ffd700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe84098e30 0x7efe841459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe837fe700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe84144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 -- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 msgr2=0x7efe84144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe84144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 -- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe84098e30 msgr2=0x7efe841459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe84098e30 0x7efe841459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 -- 172.21.15.117:0/764058031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84146370 con 0x7efe8409b700 2020-11-20T13:55:52.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe827fc700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe840b19d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7efe74002f30 tx=0x7efe74015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe73fff700 1 -- 172.21.15.117:0/764058031 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe74011b40 con 0x7efe8409b700 2020-11-20T13:55:52.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe73fff700 1 -- 172.21.15.117:0/764058031 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe74011ca0 con 0x7efe8409b700 2020-11-20T13:55:52.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe73fff700 1 -- 172.21.15.117:0/764058031 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe7401e460 con 0x7efe8409b700 2020-11-20T13:55:52.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe84145340 con 0x7efe8409b700 2020-11-20T13:55:52.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.185+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe840b3450 con 0x7efe8409b700 2020-11-20T13:55:52.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.186+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe84144c50 con 0x7efe8409b700 2020-11-20T13:55:52.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.186+0000 7efe73fff700 1 -- 172.21.15.117:0/764058031 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7efe7401a040 con 0x7efe8409b700 2020-11-20T13:55:52.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.186+0000 7efe73fff700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efe60051210 0x7efe600536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.186+0000 7efe73fff700 1 -- 172.21.15.117:0/764058031 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efe7406a130 con 0x7efe8409b700 2020-11-20T13:55:52.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.186+0000 7efe82ffd700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efe60051210 0x7efe600536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.187+0000 7efe82ffd700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efe60051210 0x7efe600536d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7efe7800c000 tx=0x7efe78008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.190+0000 7efe73fff700 1 -- 172.21.15.117:0/764058031 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7efe74042350 con 0x7efe8409b700 2020-11-20T13:55:52.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.342+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe84057050 con 0x7efe8409b700 2020-11-20T13:55:52.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.343+0000 7efe73fff700 1 -- 172.21.15.117:0/764058031 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7efe74023090 con 0x7efe8409b700 2020-11-20T13:55:52.344 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:52.344 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:52.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efe60051210 msgr2=0x7efe600536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efe60051210 0x7efe600536d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7efe7800c000 tx=0x7efe78008040).stop 2020-11-20T13:55:52.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 msgr2=0x7efe840b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe840b19d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7efe74002f30 tx=0x7efe74015000).stop 2020-11-20T13:55:52.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 shutdown_connections 2020-11-20T13:55:52.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efe60051210 0x7efe600536d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efe84143dd0 0x7efe84144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efe84098e30 0x7efe841459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 --2- 172.21.15.117:0/764058031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8409b700 0x7efe840b19d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 >> 172.21.15.117:0/764058031 conn(0x7efe8405b9f0 msgr2=0x7efe8409bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:52.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 shutdown_connections 2020-11-20T13:55:52.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.345+0000 7efe8984d700 1 -- 172.21.15.117:0/764058031 wait complete. 2020-11-20T13:55:52.356 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:55:52.357 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-20T13:55:52.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.462+0000 7fefb9b28700 1 Processor -- start 2020-11-20T13:55:52.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.462+0000 7fefb9b28700 1 -- start start 2020-11-20T13:55:52.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb9b28700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb4098e30 0x7fefb409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb9b28700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fefb40ae6f0 0x7fefb40aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb9b28700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40b5510 con 0x7fefb409b700 2020-11-20T13:55:52.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb9b28700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40b57b0 con 0x7fefb40ae6f0 2020-11-20T13:55:52.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb9b28700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40b5a50 con 0x7fefb4098e30 2020-11-20T13:55:52.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fefb40ae6f0 0x7fefb40aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb2ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fefb409b700 0x7fefb41438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59262/0 (socket says 172.21.15.117:59262) 2020-11-20T13:55:52.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fefb40ae6f0 0x7fefb40aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40102/0 (socket says 172.21.15.117:40102) 2020-11-20T13:55:52.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb2ffd700 1 -- 172.21.15.117:0/3463792353 learned_addr learned my addr 172.21.15.117:0/3463792353 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:52.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.463+0000 7fefb37fe700 1 --2- 172.21.15.117:0/3463792353 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb4098e30 0x7fefb409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb3fff700 1 -- 172.21.15.117:0/3463792353 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb4098e30 msgr2=0x7fefb409b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:52.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb3fff700 1 --2- 172.21.15.117:0/3463792353 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb4098e30 0x7fefb409b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb3fff700 1 -- 172.21.15.117:0/3463792353 >> v1:172.21.15.117:6789/0 conn(0x7fefb409b700 legacy=0x7fefb41438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb3fff700 1 -- 172.21.15.117:0/3463792353 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4099210 con 0x7fefb40ae6f0 2020-11-20T13:55:52.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb27fc700 1 -- 172.21.15.117:0/3463792353 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487865787 0 0) 0x7fefb40b5510 con 0x7fefb409b700 2020-11-20T13:55:52.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb37fe700 1 --2- 172.21.15.117:0/3463792353 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb4098e30 0x7fefb409b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:52.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb3fff700 1 --2- 172.21.15.117:0/3463792353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb40ae6f0 0x7fefb40aeb80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fefac0032b0 tx=0x7fefac0091a0).ready entity=mon.1 client_cookie=1b70c13121748a9c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb27fc700 1 -- 172.21.15.117:0/3463792353 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefac00ba50 con 0x7fefb40ae6f0 2020-11-20T13:55:52.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb27fc700 1 -- 172.21.15.117:0/3463792353 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac00bbb0 con 0x7fefb40ae6f0 2020-11-20T13:55:52.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb27fc700 1 -- 172.21.15.117:0/3463792353 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefac00bd20 con 0x7fefb40ae6f0 2020-11-20T13:55:52.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 -- 172.21.15.117:0/3463792353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb40ae6f0 msgr2=0x7fefb40aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3463792353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb40ae6f0 0x7fefb40aeb80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fefac0032b0 tx=0x7fefac0091a0).stop 2020-11-20T13:55:52.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 -- 172.21.15.117:0/3463792353 shutdown_connections 2020-11-20T13:55:52.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3463792353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb40ae6f0 0x7fefb40aeb80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3463792353 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb4098e30 0x7fefb409b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 -- 172.21.15.117:0/3463792353 >> 172.21.15.117:0/3463792353 conn(0x7fefb405b9f0 msgr2=0x7fefb405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:52.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 -- 172.21.15.117:0/3463792353 shutdown_connections 2020-11-20T13:55:52.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.464+0000 7fefb9b28700 1 -- 172.21.15.117:0/3463792353 wait complete. 2020-11-20T13:55:52.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.465+0000 7fefb9b28700 1 Processor -- start 2020-11-20T13:55:52.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.465+0000 7fefb9b28700 1 -- start start 2020-11-20T13:55:52.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb9b28700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4098e30 0x7fefb4063e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb9b28700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb409b700 0x7fefb40b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb9b28700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fefb40ae6f0 0x7fefb41409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb9b28700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb41447e0 con 0x7fefb40ae6f0 2020-11-20T13:55:52.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb9b28700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40675d0 con 0x7fefb4098e30 2020-11-20T13:55:52.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb9b28700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4064440 con 0x7fefb409b700 2020-11-20T13:55:52.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4098e30 0x7fefb4063e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4098e30 0x7fefb4063e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40106/0 (socket says 172.21.15.117:40106) 2020-11-20T13:55:52.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 -- 172.21.15.117:0/3422227200 learned_addr learned my addr 172.21.15.117:0/3422227200 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:52.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 -- 172.21.15.117:0/3422227200 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb409b700 msgr2=0x7fefb40b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb409b700 0x7fefb40b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:52.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 -- 172.21.15.117:0/3422227200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fefb40ae6f0 msgr2=0x7fefb41409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fefb40ae6f0 0x7fefb41409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:52.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.466+0000 7fefb37fe700 1 -- 172.21.15.117:0/3422227200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4144a10 con 0x7fefb4098e30 2020-11-20T13:55:52.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.467+0000 7fefb37fe700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4098e30 0x7fefb4063e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fefa4006910 tx=0x7fefa4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.467+0000 7fefb0ff9700 1 -- 172.21.15.117:0/3422227200 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefa40099b0 con 0x7fefb4098e30 2020-11-20T13:55:52.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.467+0000 7fefb0ff9700 1 -- 172.21.15.117:0/3422227200 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4009b10 con 0x7fefb4098e30 2020-11-20T13:55:52.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.467+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb4066cc0 con 0x7fefb4098e30 2020-11-20T13:55:52.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.467+0000 7fefb0ff9700 1 -- 172.21.15.117:0/3422227200 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fefa4009e70 con 0x7fefb4098e30 2020-11-20T13:55:52.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.467+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb41410d0 con 0x7fefb4098e30 2020-11-20T13:55:52.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.468+0000 7fefb0ff9700 1 -- 172.21.15.117:0/3422227200 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fefa4013480 con 0x7fefb4098e30 2020-11-20T13:55:52.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.468+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb4141380 con 0x7fefb4098e30 2020-11-20T13:55:52.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.468+0000 7fefb0ff9700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef940513c0 0x7fef94053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.468+0000 7fefb0ff9700 1 -- 172.21.15.117:0/3422227200 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fefa4069980 con 0x7fefb4098e30 2020-11-20T13:55:52.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.468+0000 7fefb2ffd700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef940513c0 0x7fef94053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.468+0000 7fefb2ffd700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef940513c0 0x7fef94053880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fefa8000b80 tx=0x7fefa8007710).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.471+0000 7fefb0ff9700 1 -- 172.21.15.117:0/3422227200 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fefa4041b40 con 0x7fefb4098e30 2020-11-20T13:55:52.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.620+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb4066fd0 con 0x7fefb4098e30 2020-11-20T13:55:52.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.621+0000 7fefb0ff9700 1 -- 172.21.15.117:0/3422227200 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fefb4066fd0 con 0x7fefb4098e30 2020-11-20T13:55:52.621 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:52.622 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:52.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.622+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef940513c0 msgr2=0x7fef94053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.622+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef940513c0 0x7fef94053880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fefa8000b80 tx=0x7fefa8007710).stop 2020-11-20T13:55:52.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4098e30 msgr2=0x7fefb4063e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4098e30 0x7fefb4063e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fefa4006910 tx=0x7fefa4008040).stop 2020-11-20T13:55:52.624 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 shutdown_connections 2020-11-20T13:55:52.624 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef940513c0 0x7fef94053880 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.624 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fefb40ae6f0 0x7fefb41409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:52.624 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4098e30 0x7fefb4063e80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.625 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 --2- 172.21.15.117:0/3422227200 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fefb409b700 0x7fefb40b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:52.625 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 >> 172.21.15.117:0/3422227200 conn(0x7fefb405b9f0 msgr2=0x7fefb40b80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:52.625 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 shutdown_connections 2020-11-20T13:55:52.625 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.623+0000 7fefb9b28700 1 -- 172.21.15.117:0/3422227200 wait complete. 2020-11-20T13:55:52.633 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:55:52.634 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:55:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.743+0000 7f7459fe9700 1 Processor -- start 2020-11-20T13:55:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.743+0000 7f7459fe9700 1 -- start start 2020-11-20T13:55:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f7459fe9700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f745409cb40 0x7f74541437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f7459fe9700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74540999d0 con 0x7f7454143c90 2020-11-20T13:55:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f7459fe9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7454099c70 con 0x7f745409bd80 2020-11-20T13:55:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f7459fe9700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74540997f0 con 0x7f745409cb40 2020-11-20T13:55:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74537fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f745409bd80 0x7f745409a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59274/0 (socket says 172.21.15.117:59274) 2020-11-20T13:55:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74537fe700 1 -- 172.21.15.117:0/2560937937 learned_addr learned my addr 172.21.15.117:0/2560937937 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f7452ffd700 1 --2- 172.21.15.117:0/2560937937 >> v2:172.21.15.85:3300/0 conn(0x7f745409cb40 0x7f74541437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702702176 0 0) 0x7f7454099c70 con 0x7f745409bd80 2020-11-20T13:55:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7440003230 con 0x7f745409bd80 2020-11-20T13:55:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1851339143 0 0) 0x7f7440003230 con 0x7f745409bd80 2020-11-20T13:55:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 >> v2:172.21.15.85:3300/0 conn(0x7f745409cb40 msgr2=0x7f74541437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 --2- 172.21.15.117:0/2560937937 >> v2:172.21.15.85:3300/0 conn(0x7f745409cb40 0x7f74541437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 >> v1:172.21.15.85:6789/0 conn(0x7f7454143c90 legacy=0x7f74540b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7454144120 con 0x7f745409bd80 2020-11-20T13:55:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f74440016d0 con 0x7f745409bd80 2020-11-20T13:55:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 >> v1:172.21.15.117:6789/0 conn(0x7f745409bd80 legacy=0x7f745409a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440003720 0x7f7440003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74400044d0 0x7f7440004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7440004e40 0x7f744000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540997f0 con 0x7f7440003720 2020-11-20T13:55:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7440004340 con 0x7f7440004e40 2020-11-20T13:55:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.744+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744000bc40 con 0x7f74400044d0 2020-11-20T13:55:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7440004e40 0x7f744000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f74537fe700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440003720 0x7f7440003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7452ffd700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74400044d0 0x7f7440004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 -- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74400044d0 msgr2=0x7f7440004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74400044d0 0x7f7440004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 -- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440003720 msgr2=0x7f7440003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440003720 0x7f7440003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 -- 172.21.15.117:0/2560937937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7440003230 con 0x7f7440004e40 2020-11-20T13:55:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 -- 172.21.15.117:0/2560937937 reap_dead start 2020-11-20T13:55:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7453fff700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7440004e40 0x7f744000b720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f745409ad40 tx=0x7f744c007e50).ready entity=mon.1 client_cookie=26155d2ff0e0c16 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f744c00c960 con 0x7f7440004e40 2020-11-20T13:55:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c00cac0 con 0x7f7440004e40 2020-11-20T13:55:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f74527fc700 1 -- 172.21.15.117:0/2560937937 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f744c00ce20 con 0x7f7440004e40 2020-11-20T13:55:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7459fe9700 1 -- 172.21.15.117:0/2560937937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7440004e40 msgr2=0x7f744000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.745+0000 7f7459fe9700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7440004e40 0x7f744000b720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f745409ad40 tx=0x7f744c007e50).stop 2020-11-20T13:55:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.746+0000 7f7459fe9700 1 -- 172.21.15.117:0/2560937937 shutdown_connections 2020-11-20T13:55:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.746+0000 7f7459fe9700 1 --2- 172.21.15.117:0/2560937937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7440004e40 0x7f744000b720 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.746+0000 7f7459fe9700 1 -- 172.21.15.117:0/2560937937 >> 172.21.15.117:0/2560937937 conn(0x7f74540b9770 msgr2=0x7f74540b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.746+0000 7f7459fe9700 1 -- 172.21.15.117:0/2560937937 shutdown_connections 2020-11-20T13:55:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.746+0000 7f7459fe9700 1 -- 172.21.15.117:0/2560937937 wait complete. 2020-11-20T13:55:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.746+0000 7f7459fe9700 1 Processor -- start 2020-11-20T13:55:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7459fe9700 1 -- start start 2020-11-20T13:55:52.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7459fe9700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440004e40 0x7f74540b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7459fe9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74540b10b0 0x7f74540aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7459fe9700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7454153680 0x7f74540b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7459fe9700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540b34f0 con 0x7f7440004e40 2020-11-20T13:55:52.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7459fe9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7454153c00 con 0x7f74540b10b0 2020-11-20T13:55:52.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7459fe9700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7454065d70 con 0x7f7454153680 2020-11-20T13:55:52.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7452ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74540b10b0 0x7f74540aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7452ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74540b10b0 0x7f74540aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40128/0 (socket says 172.21.15.117:40128) 2020-11-20T13:55:52.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7452ffd700 1 -- 172.21.15.117:0/4054451922 learned_addr learned my addr 172.21.15.117:0/4054451922 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:52.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f7453fff700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7454153680 0x7f74540b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.747+0000 7f74537fe700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440004e40 0x7f74540b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7452ffd700 1 -- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7454153680 msgr2=0x7f74540b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7452ffd700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7454153680 0x7f74540b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7452ffd700 1 -- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440004e40 msgr2=0x7f74540b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7452ffd700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440004e40 0x7f74540b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7452ffd700 1 -- 172.21.15.117:0/4054451922 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7454066010 con 0x7f74540b10b0 2020-11-20T13:55:52.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7452ffd700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74540b10b0 0x7f74540aff80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f7454152c30 tx=0x7f7448008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7450ff9700 1 -- 172.21.15.117:0/4054451922 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f744800f910 con 0x7f74540b10b0 2020-11-20T13:55:52.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7450ff9700 1 -- 172.21.15.117:0/4054451922 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744800fa70 con 0x7f74540b10b0 2020-11-20T13:55:52.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74540b0960 con 0x7f74540b10b0 2020-11-20T13:55:52.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7450ff9700 1 -- 172.21.15.117:0/4054451922 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f744800fdd0 con 0x7f74540b10b0 2020-11-20T13:55:52.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.748+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74540aedb0 con 0x7f74540b10b0 2020-11-20T13:55:52.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.749+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7454153a60 con 0x7f74540b10b0 2020-11-20T13:55:52.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.749+0000 7f7450ff9700 1 -- 172.21.15.117:0/4054451922 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f7448014440 con 0x7f74540b10b0 2020-11-20T13:55:52.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.749+0000 7f7450ff9700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7434051230 0x7f74340536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:52.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.749+0000 7f7450ff9700 1 -- 172.21.15.117:0/4054451922 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f74480699d0 con 0x7f74540b10b0 2020-11-20T13:55:52.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.749+0000 7f74537fe700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7434051230 0x7f74340536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:52.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.750+0000 7f74537fe700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7434051230 0x7f74340536f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7444005000 tx=0x7f7454099b60).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:52.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.753+0000 7f7450ff9700 1 -- 172.21.15.117:0/4054451922 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f7454153a60 con 0x7f74540b10b0 2020-11-20T13:55:52.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.902+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7454153a60 con 0x7f74540b10b0 2020-11-20T13:55:52.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.902+0000 7f7450ff9700 1 -- 172.21.15.117:0/4054451922 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7454153a60 con 0x7f74540b10b0 2020-11-20T13:55:52.903 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:55:52.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7434051230 msgr2=0x7f74340536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7434051230 0x7f74340536f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f7444005000 tx=0x7f7454099b60).stop 2020-11-20T13:55:52.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74540b10b0 msgr2=0x7f74540aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:52.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74540b10b0 0x7f74540aff80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f7454152c30 tx=0x7f7448008040).stop 2020-11-20T13:55:52.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 shutdown_connections 2020-11-20T13:55:52.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7434051230 0x7f74340536f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7454153680 0x7f74540b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7440004e40 0x7f74540b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 --2- 172.21.15.117:0/4054451922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74540b10b0 0x7f74540aff80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:52.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 >> 172.21.15.117:0/4054451922 conn(0x7f74540b9770 msgr2=0x7f745409f090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:52.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 shutdown_connections 2020-11-20T13:55:52.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:52.904+0000 7f7459fe9700 1 -- 172.21.15.117:0/4054451922 wait complete. 2020-11-20T13:55:52.914 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:55:53.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.024+0000 7f0622374700 1 Processor -- start 2020-11-20T13:55:53.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.024+0000 7f0622374700 1 -- start start 2020-11-20T13:55:53.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0622374700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f061c09bd80 0x7f061c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0622374700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c0999d0 con 0x7f061c09a110 2020-11-20T13:55:53.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0622374700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c099c70 con 0x7f061c143eb0 2020-11-20T13:55:53.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0622374700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061c09c160 con 0x7f061c09bd80 2020-11-20T13:55:53.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f061c09bd80 0x7f061c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0620911700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f061c143eb0 0x7f061c0b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59294/0 (socket says 172.21.15.117:59294) 2020-11-20T13:55:53.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0620911700 1 -- 172.21.15.117:0/3150726629 learned_addr learned my addr 172.21.15.117:0/3150726629 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:53.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061bfff700 1 -- 172.21.15.117:0/3150726629 >> v1:172.21.15.117:6789/0 conn(0x7f061c143eb0 legacy=0x7f061c0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061bfff700 1 -- 172.21.15.117:0/3150726629 >> v1:172.21.15.85:6789/0 conn(0x7f061c09a110 legacy=0x7f061c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061bfff700 1 -- 172.21.15.117:0/3150726629 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c09a5c0 con 0x7f061c09bd80 2020-11-20T13:55:53.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061bfff700 1 --2- 172.21.15.117:0/3150726629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c09cb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f060c002f30 tx=0x7f060c006dc0).ready entity=mon.1 client_cookie=1cffc57cc3fe6c88 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061affd700 1 -- 172.21.15.117:0/3150726629 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f060c0097f0 con 0x7f061c09bd80 2020-11-20T13:55:53.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061affd700 1 -- 172.21.15.117:0/3150726629 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c009950 con 0x7f061c09bd80 2020-11-20T13:55:53.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f061affd700 1 -- 172.21.15.117:0/3150726629 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f060c009cb0 con 0x7f061c09bd80 2020-11-20T13:55:53.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0622374700 1 -- 172.21.15.117:0/3150726629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 msgr2=0x7f061c09cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.025+0000 7f0622374700 1 --2- 172.21.15.117:0/3150726629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c09cb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f060c002f30 tx=0x7f060c006dc0).stop 2020-11-20T13:55:53.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.026+0000 7f0622374700 1 -- 172.21.15.117:0/3150726629 shutdown_connections 2020-11-20T13:55:53.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.026+0000 7f0622374700 1 --2- 172.21.15.117:0/3150726629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c09cb40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.026+0000 7f0622374700 1 -- 172.21.15.117:0/3150726629 >> 172.21.15.117:0/3150726629 conn(0x7f061c0b9770 msgr2=0x7f061c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.026+0000 7f0622374700 1 -- 172.21.15.117:0/3150726629 shutdown_connections 2020-11-20T13:55:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.026+0000 7f0622374700 1 -- 172.21.15.117:0/3150726629 wait complete. 2020-11-20T13:55:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.027+0000 7f0622374700 1 Processor -- start 2020-11-20T13:55:53.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.027+0000 7f0622374700 1 -- start start 2020-11-20T13:55:53.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f061c09a110 0x7f061c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f061c143eb0 0x7f061c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c15aba0 con 0x7f061c143eb0 2020-11-20T13:55:53.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1567e0 con 0x7f061c09bd80 2020-11-20T13:55:53.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c156a80 con 0x7f061c09a110 2020-11-20T13:55:53.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c15a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40140/0 (socket says 172.21.15.117:40140) 2020-11-20T13:55:53.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 -- 172.21.15.117:0/831505582 learned_addr learned my addr 172.21.15.117:0/831505582 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:53.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0620911700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f061c143eb0 0x7f061c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 -- 172.21.15.117:0/831505582 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f061c09a110 msgr2=0x7f061c1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f061c09a110 0x7f061c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:53.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 -- 172.21.15.117:0/831505582 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f061c143eb0 msgr2=0x7f061c0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f061c143eb0 0x7f061c0b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 -- 172.21.15.117:0/831505582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c156d20 con 0x7f061c09bd80 2020-11-20T13:55:53.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f061b7fe700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c15a6b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f061c09afe0 tx=0x7f0610007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f06197fa700 1 -- 172.21.15.117:0/831505582 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0610010510 con 0x7f061c09bd80 2020-11-20T13:55:53.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f06197fa700 1 -- 172.21.15.117:0/831505582 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0610010670 con 0x7f061c09bd80 2020-11-20T13:55:53.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c09f210 con 0x7f061c09bd80 2020-11-20T13:55:53.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f06197fa700 1 -- 172.21.15.117:0/831505582 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06100107e0 con 0x7f061c09bd80 2020-11-20T13:55:53.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.028+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c0b3210 con 0x7f061c09bd80 2020-11-20T13:55:53.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.029+0000 7f06197fa700 1 -- 172.21.15.117:0/831505582 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0610010070 con 0x7f061c09bd80 2020-11-20T13:55:53.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.029+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c156eb0 con 0x7f061c09bd80 2020-11-20T13:55:53.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.030+0000 7f06197fa700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05fc051320 0x7f05fc0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.030+0000 7f06197fa700 1 -- 172.21.15.117:0/831505582 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0610068a50 con 0x7f061c09bd80 2020-11-20T13:55:53.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.030+0000 7f061bfff700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05fc051320 0x7f05fc0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.030+0000 7f061bfff700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05fc051320 0x7f05fc0537e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f060c002f30 tx=0x7f060c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.033+0000 7f06197fa700 1 -- 172.21.15.117:0/831505582 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0610040c70 con 0x7f061c09bd80 2020-11-20T13:55:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.183+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f061c09f3a0 con 0x7f061c09bd80 2020-11-20T13:55:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.184+0000 7f06197fa700 1 -- 172.21.15.117:0/831505582 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f061c09f3a0 con 0x7f061c09bd80 2020-11-20T13:55:53.184 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:55:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05fc051320 msgr2=0x7f05fc0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05fc051320 0x7f05fc0537e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f060c002f30 tx=0x7f060c015000).stop 2020-11-20T13:55:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 msgr2=0x7f061c15a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c15a6b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f061c09afe0 tx=0x7f0610007b90).stop 2020-11-20T13:55:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 shutdown_connections 2020-11-20T13:55:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f05fc051320 0x7f05fc0537e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f061c09a110 0x7f061c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:55:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f061c143eb0 0x7f061c0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 --2- 172.21.15.117:0/831505582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f061c09bd80 0x7f061c15a6b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 >> 172.21.15.117:0/831505582 conn(0x7f061c0b9770 msgr2=0x7f061c062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 shutdown_connections 2020-11-20T13:55:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.185+0000 7f0622374700 1 -- 172.21.15.117:0/831505582 wait complete. 2020-11-20T13:55:53.196 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:55:53.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.304+0000 7f8223cfd700 1 Processor -- start 2020-11-20T13:55:53.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.304+0000 7f8223cfd700 1 -- start start 2020-11-20T13:55:53.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8223cfd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f821c09bd80 0x7f821c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8223cfd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09a110 0x7f821c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8223cfd700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c0999d0 con 0x7f821c143c90 2020-11-20T13:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8223cfd700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c099c70 con 0x7f821c09bd80 2020-11-20T13:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8223cfd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c144120 con 0x7f821c09a110 2020-11-20T13:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f821c09bd80 0x7f821c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f821c09bd80 0x7f821c09cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40146/0 (socket says 172.21.15.117:40146) 2020-11-20T13:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 -- 172.21.15.117:0/1428446948 learned_addr learned my addr 172.21.15.117:0/1428446948 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 -- 172.21.15.117:0/1428446948 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09a110 msgr2=0x7f821c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221298700 1 --2- 172.21.15.117:0/1428446948 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09a110 0x7f821c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 --2- 172.21.15.117:0/1428446948 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09a110 0x7f821c1437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 -- 172.21.15.117:0/1428446948 >> v1:172.21.15.85:6789/0 conn(0x7f821c143c90 legacy=0x7f821c0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 -- 172.21.15.117:0/1428446948 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c09c160 con 0x7f821c09bd80 2020-11-20T13:55:53.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.305+0000 7f8221a99700 1 --2- 172.21.15.117:0/1428446948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c09bd80 0x7f821c09cb40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f821c09b9c0 tx=0x7f820c006e80).ready entity=mon.1 client_cookie=2887327365e65cc2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8220a97700 1 -- 172.21.15.117:0/1428446948 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f820c009770 con 0x7f821c09bd80 2020-11-20T13:55:53.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8220a97700 1 -- 172.21.15.117:0/1428446948 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c0098d0 con 0x7f821c09bd80 2020-11-20T13:55:53.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8220a97700 1 -- 172.21.15.117:0/1428446948 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f820c009a40 con 0x7f821c09bd80 2020-11-20T13:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 -- 172.21.15.117:0/1428446948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c09bd80 msgr2=0x7f821c09cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1428446948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c09bd80 0x7f821c09cb40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f821c09b9c0 tx=0x7f820c006e80).stop 2020-11-20T13:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 -- 172.21.15.117:0/1428446948 shutdown_connections 2020-11-20T13:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1428446948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c09bd80 0x7f821c09cb40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1428446948 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09a110 0x7f821c1437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 -- 172.21.15.117:0/1428446948 >> 172.21.15.117:0/1428446948 conn(0x7f821c0b9770 msgr2=0x7f821c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:53.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 -- 172.21.15.117:0/1428446948 shutdown_connections 2020-11-20T13:55:53.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.306+0000 7f8223cfd700 1 -- 172.21.15.117:0/1428446948 wait complete. 2020-11-20T13:55:53.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.307+0000 7f8223cfd700 1 Processor -- start 2020-11-20T13:55:53.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.307+0000 7f8223cfd700 1 -- start start 2020-11-20T13:55:53.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f821c09a110 0x7f821c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09bd80 0x7f821c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c143c90 0x7f821c05f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c155010 con 0x7f821c09a110 2020-11-20T13:55:53.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c067b70 con 0x7f821c143c90 2020-11-20T13:55:53.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c067630 con 0x7f821c09bd80 2020-11-20T13:55:53.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c143c90 0x7f821c05f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c143c90 0x7f821c05f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40154/0 (socket says 172.21.15.117:40154) 2020-11-20T13:55:53.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 -- 172.21.15.117:0/1773818609 learned_addr learned my addr 172.21.15.117:0/1773818609 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:53.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8221a99700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f821c09a110 0x7f821c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 -- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09bd80 msgr2=0x7f821c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8221298700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09bd80 0x7f821c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09bd80 0x7f821c154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 -- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f821c09a110 msgr2=0x7f821c0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f821c09a110 0x7f821c0b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 -- 172.21.15.117:0/1773818609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0678d0 con 0x7f821c143c90 2020-11-20T13:55:53.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8221a99700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f821c09a110 0x7f821c0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:53.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f822229a700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c143c90 0x7f821c05f1e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f82140022d0 tx=0x7f8214007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f820affd700 1 -- 172.21.15.117:0/1773818609 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f821400a7b0 con 0x7f821c143c90 2020-11-20T13:55:53.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f820affd700 1 -- 172.21.15.117:0/1773818609 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821400a910 con 0x7f821c143c90 2020-11-20T13:55:53.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c067320 con 0x7f821c143c90 2020-11-20T13:55:53.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f820affd700 1 -- 172.21.15.117:0/1773818609 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f821400ac70 con 0x7f821c143c90 2020-11-20T13:55:53.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.308+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c05f6d0 con 0x7f821c143c90 2020-11-20T13:55:53.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.310+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c157600 con 0x7f821c143c90 2020-11-20T13:55:53.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.310+0000 7f820affd700 1 -- 172.21.15.117:0/1773818609 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8214019030 con 0x7f821c143c90 2020-11-20T13:55:53.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.310+0000 7f820affd700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f81fc051320 0x7f81fc0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.310+0000 7f820affd700 1 -- 172.21.15.117:0/1773818609 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f82140688d0 con 0x7f821c143c90 2020-11-20T13:55:53.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.310+0000 7f8221a99700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f81fc051320 0x7f81fc0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.310+0000 7f8221a99700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f81fc051320 0x7f81fc0537e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f820c017000 tx=0x7f820c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.313+0000 7f820affd700 1 -- 172.21.15.117:0/1773818609 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8214040af0 con 0x7f821c143c90 2020-11-20T13:55:53.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.463+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f821c057050 con 0x7f821c143c90 2020-11-20T13:55:53.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.463+0000 7f820affd700 1 -- 172.21.15.117:0/1773818609 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8214044110 con 0x7f821c143c90 2020-11-20T13:55:53.464 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:55:53.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f81fc051320 msgr2=0x7f81fc0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f81fc051320 0x7f81fc0537e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f820c017000 tx=0x7f820c016000).stop 2020-11-20T13:55:53.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c143c90 msgr2=0x7f821c05f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c143c90 0x7f821c05f1e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f82140022d0 tx=0x7f8214007f10).stop 2020-11-20T13:55:53.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 shutdown_connections 2020-11-20T13:55:53.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f81fc051320 0x7f81fc0537e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f821c143c90 0x7f821c05f1e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f821c09a110 0x7f821c0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 --2- 172.21.15.117:0/1773818609 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f821c09bd80 0x7f821c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 >> 172.21.15.117:0/1773818609 conn(0x7f821c0b9770 msgr2=0x7f821c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:53.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 shutdown_connections 2020-11-20T13:55:53.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.465+0000 7f8223cfd700 1 -- 172.21.15.117:0/1773818609 wait complete. 2020-11-20T13:55:53.475 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:55:53.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.583+0000 7fd49b59e700 1 Processor -- start 2020-11-20T13:55:53.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.583+0000 7fd49b59e700 1 -- start start 2020-11-20T13:55:53.585 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.584+0000 7fd49b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd49c09cdd0 0x7fd49c147c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.584+0000 7fd49b59e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd49c05ee90 0x7fd49c067200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.584+0000 7fd49b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49c066890 con 0x7fd49c146840 2020-11-20T13:55:53.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.584+0000 7fd49b59e700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c148140 con 0x7fd49c05ee90 2020-11-20T13:55:53.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.584+0000 7fd49b59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c066ac0 con 0x7fd49c09cdd0 2020-11-20T13:55:53.587 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.584+0000 7fd49a59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd49c09cdd0 0x7fd49c147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.587 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49a59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd49c09cdd0 0x7fd49c147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40158/0 (socket says 172.21.15.117:40158) 2020-11-20T13:55:53.587 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.584+0000 7fd49ad9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd49c146840 0x7fd49c09e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59318/0 (socket says 172.21.15.117:59318) 2020-11-20T13:55:53.587 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49a59c700 1 -- 172.21.15.117:0/55783656 learned_addr learned my addr 172.21.15.117:0/55783656 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:53.587 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd499d9b700 1 --2- 172.21.15.117:0/55783656 >> v2:172.21.15.85:3300/0 conn(0x7fd49c05ee90 0x7fd49c067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49a59c700 1 -- 172.21.15.117:0/55783656 >> v2:172.21.15.85:3300/0 conn(0x7fd49c05ee90 msgr2=0x7fd49c067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:53.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49a59c700 1 --2- 172.21.15.117:0/55783656 >> v2:172.21.15.85:3300/0 conn(0x7fd49c05ee90 0x7fd49c067200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49a59c700 1 -- 172.21.15.117:0/55783656 >> v1:172.21.15.117:6789/0 conn(0x7fd49c146840 legacy=0x7fd49c09e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49959a700 1 -- 172.21.15.117:0/55783656 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194995044 0 0) 0x7fd49c066890 con 0x7fd49c146840 2020-11-20T13:55:53.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49a59c700 1 -- 172.21.15.117:0/55783656 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c066d60 con 0x7fd49c09cdd0 2020-11-20T13:55:53.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49a59c700 1 --2- 172.21.15.117:0/55783656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c147c50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd490002f30 tx=0x7fd490006e80).ready entity=mon.1 client_cookie=992d8fad17b377ac server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49959a700 1 -- 172.21.15.117:0/55783656 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd4900097f0 con 0x7fd49c09cdd0 2020-11-20T13:55:53.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49959a700 1 -- 172.21.15.117:0/55783656 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd490009950 con 0x7fd49c09cdd0 2020-11-20T13:55:53.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49959a700 1 -- 172.21.15.117:0/55783656 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd490009cb0 con 0x7fd49c09cdd0 2020-11-20T13:55:53.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49b59e700 1 -- 172.21.15.117:0/55783656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 msgr2=0x7fd49c147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.585+0000 7fd49b59e700 1 --2- 172.21.15.117:0/55783656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c147c50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fd490002f30 tx=0x7fd490006e80).stop 2020-11-20T13:55:53.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.586+0000 7fd49b59e700 1 -- 172.21.15.117:0/55783656 shutdown_connections 2020-11-20T13:55:53.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.586+0000 7fd49b59e700 1 --2- 172.21.15.117:0/55783656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c147c50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.586+0000 7fd49b59e700 1 --2- 172.21.15.117:0/55783656 >> v2:172.21.15.85:3300/0 conn(0x7fd49c05ee90 0x7fd49c067200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.586+0000 7fd49b59e700 1 -- 172.21.15.117:0/55783656 >> 172.21.15.117:0/55783656 conn(0x7fd49c0bd890 msgr2=0x7fd49c0bdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:53.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.586+0000 7fd49b59e700 1 -- 172.21.15.117:0/55783656 shutdown_connections 2020-11-20T13:55:53.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.586+0000 7fd49b59e700 1 -- 172.21.15.117:0/55783656 wait complete. 2020-11-20T13:55:53.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 Processor -- start 2020-11-20T13:55:53.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 -- start start 2020-11-20T13:55:53.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd49c05ee90 0x7fd49c0bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c0b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd49c146840 0x7fd49c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0b43d0 con 0x7fd49c05ee90 2020-11-20T13:55:53.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0b6540 con 0x7fd49c09cdd0 2020-11-20T13:55:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49b59e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0b67e0 con 0x7fd49c146840 2020-11-20T13:55:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd499d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c0b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd499d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c0b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40164/0 (socket says 172.21.15.117:40164) 2020-11-20T13:55:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd499d9b700 1 -- 172.21.15.117:0/3311908799 learned_addr learned my addr 172.21.15.117:0/3311908799 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49a59c700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd49c05ee90 0x7fd49c0bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.587+0000 7fd49ad9d700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd49c146840 0x7fd49c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd499d9b700 1 -- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd49c146840 msgr2=0x7fd49c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd499d9b700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd49c146840 0x7fd49c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd499d9b700 1 -- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd49c05ee90 msgr2=0x7fd49c0bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd499d9b700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd49c05ee90 0x7fd49c0bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd499d9b700 1 -- 172.21.15.117:0/3311908799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c0b6a80 con 0x7fd49c09cdd0 2020-11-20T13:55:53.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd49a59c700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd49c05ee90 0x7fd49c0bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:53.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd49ad9d700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd49c146840 0x7fd49c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:53.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd499d9b700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c0b6f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd48c007000 tx=0x7fd48c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd48b7fe700 1 -- 172.21.15.117:0/3311908799 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd48c010580 con 0x7fd49c09cdd0 2020-11-20T13:55:53.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd48b7fe700 1 -- 172.21.15.117:0/3311908799 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c0106e0 con 0x7fd49c09cdd0 2020-11-20T13:55:53.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c0b2930 con 0x7fd49c09cdd0 2020-11-20T13:55:53.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd48b7fe700 1 -- 172.21.15.117:0/3311908799 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd48c010a40 con 0x7fd49c09cdd0 2020-11-20T13:55:53.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.588+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c0b4ec0 con 0x7fd49c09cdd0 2020-11-20T13:55:53.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.589+0000 7fd48b7fe700 1 -- 172.21.15.117:0/3311908799 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd48c010ba0 con 0x7fd49c09cdd0 2020-11-20T13:55:53.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.589+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c0b6c10 con 0x7fd49c09cdd0 2020-11-20T13:55:53.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.589+0000 7fd48b7fe700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd478051210 0x7fd4780536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.589+0000 7fd48b7fe700 1 -- 172.21.15.117:0/3311908799 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd48c069270 con 0x7fd49c09cdd0 2020-11-20T13:55:53.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.589+0000 7fd49a59c700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd478051210 0x7fd4780536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.589+0000 7fd49a59c700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd478051210 0x7fd4780536d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd49001a000 tx=0x7fd490015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.593+0000 7fd48b7fe700 1 -- 172.21.15.117:0/3311908799 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd48c041490 con 0x7fd49c09cdd0 2020-11-20T13:55:53.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.743+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd49c057050 con 0x7fd49c09cdd0 2020-11-20T13:55:53.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.743+0000 7fd48b7fe700 1 -- 172.21.15.117:0/3311908799 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd48c044ab0 con 0x7fd49c09cdd0 2020-11-20T13:55:53.744 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:55:53.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.744+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd478051210 msgr2=0x7fd4780536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.744+0000 7fd49b59e700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd478051210 0x7fd4780536d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd49001a000 tx=0x7fd490015000).stop 2020-11-20T13:55:53.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.744+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 msgr2=0x7fd49c0b6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.744+0000 7fd49b59e700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c0b6f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd48c007000 tx=0x7fd48c008040).stop 2020-11-20T13:55:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 shutdown_connections 2020-11-20T13:55:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd478051210 0x7fd4780536d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd49c146840 0x7fd49c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd49c05ee90 0x7fd49c0bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 --2- 172.21.15.117:0/3311908799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49c09cdd0 0x7fd49c0b6f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 >> 172.21.15.117:0/3311908799 conn(0x7fd49c0bd890 msgr2=0x7fd49c145c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:53.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 shutdown_connections 2020-11-20T13:55:53.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.745+0000 7fd49b59e700 1 -- 172.21.15.117:0/3311908799 wait complete. 2020-11-20T13:55:53.757 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-20T13:55:53.758 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-20T13:55:53.758 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-11-20T13:55:53.865 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.864+0000 7f1ba1009700 1 Processor -- start 2020-11-20T13:55:53.865 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.865+0000 7f1ba1009700 1 -- start start 2020-11-20T13:55:53.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.865+0000 7f1ba1009700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0b5860 0x7f1b9c098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.865+0000 7f1ba1009700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b9c099080 0x7f1b9c099fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.865+0000 7f1ba1009700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c0af3a0 con 0x7f1b9c0ae8b0 2020-11-20T13:55:53.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.865+0000 7f1ba1009700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b4e80 con 0x7f1b9c099080 2020-11-20T13:55:53.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.865+0000 7f1ba1009700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b5120 con 0x7f1b9c0b5860 2020-11-20T13:55:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.865+0000 7f1b9b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b9c099080 0x7f1b9c099fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b9c099080 0x7f1b9c099fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40170/0 (socket says 172.21.15.117:40170) 2020-11-20T13:55:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9ad9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1b9c0ae8b0 0x7f1b9c0aec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59330/0 (socket says 172.21.15.117:59330) 2020-11-20T13:55:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9b59e700 1 -- 172.21.15.117:0/729679717 learned_addr learned my addr 172.21.15.117:0/729679717 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9a59c700 1 --2- 172.21.15.117:0/729679717 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0b5860 0x7f1b9c098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9b59e700 1 -- 172.21.15.117:0/729679717 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0b5860 msgr2=0x7f1b9c098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9b59e700 1 --2- 172.21.15.117:0/729679717 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0b5860 0x7f1b9c098b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9b59e700 1 -- 172.21.15.117:0/729679717 >> v1:172.21.15.117:6789/0 conn(0x7f1b9c0ae8b0 legacy=0x7f1b9c0aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b99d9b700 1 -- 172.21.15.117:0/729679717 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850506603 0 0) 0x7f1b9c0af3a0 con 0x7f1b9c0ae8b0 2020-11-20T13:55:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9b59e700 1 -- 172.21.15.117:0/729679717 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0b53c0 con 0x7f1b9c099080 2020-11-20T13:55:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9a59c700 1 --2- 172.21.15.117:0/729679717 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0b5860 0x7f1b9c098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b9b59e700 1 --2- 172.21.15.117:0/729679717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c099080 0x7f1b9c099fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1b940032b0 tx=0x7f1b94009270).ready entity=mon.1 client_cookie=cd840c0c7a594469 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b99d9b700 1 -- 172.21.15.117:0/729679717 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b9400bc00 con 0x7f1b9c099080 2020-11-20T13:55:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b99d9b700 1 -- 172.21.15.117:0/729679717 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400bd60 con 0x7f1b9c099080 2020-11-20T13:55:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1b99d9b700 1 -- 172.21.15.117:0/729679717 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b94011400 con 0x7f1b9c099080 2020-11-20T13:55:53.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1ba1009700 1 -- 172.21.15.117:0/729679717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c099080 msgr2=0x7f1b9c099fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.866+0000 7f1ba1009700 1 --2- 172.21.15.117:0/729679717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c099080 0x7f1b9c099fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1b940032b0 tx=0x7f1b94009270).stop 2020-11-20T13:55:53.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.867+0000 7f1ba1009700 1 -- 172.21.15.117:0/729679717 shutdown_connections 2020-11-20T13:55:53.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.867+0000 7f1ba1009700 1 --2- 172.21.15.117:0/729679717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c099080 0x7f1b9c099fa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.867+0000 7f1ba1009700 1 --2- 172.21.15.117:0/729679717 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0b5860 0x7f1b9c098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.867+0000 7f1ba1009700 1 -- 172.21.15.117:0/729679717 >> 172.21.15.117:0/729679717 conn(0x7f1b9c05b9f0 msgr2=0x7f1b9c05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:53.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.867+0000 7f1ba1009700 1 -- 172.21.15.117:0/729679717 shutdown_connections 2020-11-20T13:55:53.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.867+0000 7f1ba1009700 1 -- 172.21.15.117:0/729679717 wait complete. 2020-11-20T13:55:53.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 Processor -- start 2020-11-20T13:55:53.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 -- start start 2020-11-20T13:55:53.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1b9c099080 0x7f1b9c142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0ae8b0 0x7f1b9c09ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c0b5860 0x7f1b9c13b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c13dc90 con 0x7f1b9c099080 2020-11-20T13:55:53.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c13df30 con 0x7f1b9c0b5860 2020-11-20T13:55:53.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1ba1009700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c13e1d0 con 0x7f1b9c0ae8b0 2020-11-20T13:55:53.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1b9b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c0b5860 0x7f1b9c13b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1b9b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c0b5860 0x7f1b9c13b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40178/0 (socket says 172.21.15.117:40178) 2020-11-20T13:55:53.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.868+0000 7f1b9b59e700 1 -- 172.21.15.117:0/2950059300 learned_addr learned my addr 172.21.15.117:0/2950059300 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:53.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9a59c700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0ae8b0 0x7f1b9c09ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9ad9d700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1b9c099080 0x7f1b9c142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9b59e700 1 -- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0ae8b0 msgr2=0x7f1b9c09ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9b59e700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0ae8b0 0x7f1b9c09ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9b59e700 1 -- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1b9c099080 msgr2=0x7f1b9c142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:53.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9b59e700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1b9c099080 0x7f1b9c142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:53.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9b59e700 1 -- 172.21.15.117:0/2950059300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c140c10 con 0x7f1b9c0b5860 2020-11-20T13:55:53.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9a59c700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0ae8b0 0x7f1b9c09ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:53.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9ad9d700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1b9c099080 0x7f1b9c142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:53.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b9b59e700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c0b5860 0x7f1b9c13b9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f1b940032b0 tx=0x7f1b94016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b93fff700 1 -- 172.21.15.117:0/2950059300 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b940107a0 con 0x7f1b9c0b5860 2020-11-20T13:55:53.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b93fff700 1 -- 172.21.15.117:0/2950059300 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b94010900 con 0x7f1b9c0b5860 2020-11-20T13:55:53.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c13d7f0 con 0x7f1b9c0b5860 2020-11-20T13:55:53.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1b93fff700 1 -- 172.21.15.117:0/2950059300 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b94010c60 con 0x7f1b9c0b5860 2020-11-20T13:55:53.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.869+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c13c0d0 con 0x7f1b9c0b5860 2020-11-20T13:55:53.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.870+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c13e390 con 0x7f1b9c0b5860 2020-11-20T13:55:53.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.870+0000 7f1b93fff700 1 -- 172.21.15.117:0/2950059300 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1b94026030 con 0x7f1b9c0b5860 2020-11-20T13:55:53.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.870+0000 7f1b93fff700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1b80051210 0x7f1b800536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:53.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.870+0000 7f1b93fff700 1 -- 172.21.15.117:0/2950059300 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1b9406a5a0 con 0x7f1b9c0b5860 2020-11-20T13:55:53.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.870+0000 7f1b9ad9d700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1b80051210 0x7f1b800536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:53.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.871+0000 7f1b9ad9d700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1b80051210 0x7f1b800536d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1b84008000 tx=0x7f1b84009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:53.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:53.874+0000 7f1b93fff700 1 -- 172.21.15.117:0/2950059300 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1b94042720 con 0x7f1b9c0b5860 2020-11-20T13:55:54.062 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.062+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1b9c13daa0 con 0x7f1b9c0b5860 2020-11-20T13:55:54.064 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.064+0000 7f1b93fff700 1 -- 172.21.15.117:0/2950059300 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+126775 (secure 0 0 0) 0x7f1b94024090 con 0x7f1b9c0b5860 2020-11-20T13:55:54.066 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:54.067 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1b80051210 msgr2=0x7f1b800536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.067 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1b80051210 0x7f1b800536d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1b84008000 tx=0x7f1b84009040).stop 2020-11-20T13:55:54.067 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c0b5860 msgr2=0x7f1b9c13b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c0b5860 0x7f1b9c13b9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f1b940032b0 tx=0x7f1b94016000).stop 2020-11-20T13:55:54.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 shutdown_connections 2020-11-20T13:55:54.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1b80051210 0x7f1b800536d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b9c0b5860 0x7f1b9c13b9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1b9c099080 0x7f1b9c142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 --2- 172.21.15.117:0/2950059300 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1b9c0ae8b0 0x7f1b9c09ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 >> 172.21.15.117:0/2950059300 conn(0x7f1b9c05b9f0 msgr2=0x7f1b9c0988a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 shutdown_connections 2020-11-20T13:55:54.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.066+0000 7f1ba1009700 1 -- 172.21.15.117:0/2950059300 wait complete. 2020-11-20T13:55:54.077 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":4,"active_gid":4111,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":22275},{"type":"v1","addr":"172.21.15.117:6801","nonce":22275}]},"active_addr":"172.21.15.117:6801/22275","active_change":"2020-11-20T13:55:34.130370+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":2039890261}]},{"addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1620220017}]},{"addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":603549421}]}]}} 2020-11-20T13:55:54.080 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-20T13:55:54.081 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-20T13:55:54.081 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:55:54.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.187+0000 7f9108996700 1 Processor -- start 2020-11-20T13:55:54.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.187+0000 7f9108996700 1 -- start start 2020-11-20T13:55:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9108996700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9104098e30 0x7f910409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9108996700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f910409b700 0x7f91041438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9108996700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91040b5510 con 0x7f9104143dd0 2020-11-20T13:55:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9108996700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040b57b0 con 0x7f910409b700 2020-11-20T13:55:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9108996700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91040b5a50 con 0x7f9104098e30 2020-11-20T13:55:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9101d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f910409b700 0x7f91041438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9102d9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9104143dd0 0x7f91040b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59342/0 (socket says 172.21.15.117:59342) 2020-11-20T13:55:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9102d9d700 1 -- 172.21.15.117:0/2633478874 learned_addr learned my addr 172.21.15.117:0/2633478874 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:54.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9101d9b700 1 --2- 172.21.15.117:0/2633478874 >> v2:172.21.15.117:3300/0 conn(0x7f910409b700 0x7f91041438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40182/0 (socket says 172.21.15.117:40182) 2020-11-20T13:55:54.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f910259c700 1 --2- 172.21.15.117:0/2633478874 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9104098e30 0x7f910409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9101d9b700 1 -- 172.21.15.117:0/2633478874 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9104098e30 msgr2=0x7f910409b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.188+0000 7f9101d9b700 1 --2- 172.21.15.117:0/2633478874 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9104098e30 0x7f910409b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9101d9b700 1 -- 172.21.15.117:0/2633478874 >> v1:172.21.15.117:6789/0 conn(0x7f9104143dd0 legacy=0x7f91040b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9101d9b700 1 -- 172.21.15.117:0/2633478874 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104099210 con 0x7f910409b700 2020-11-20T13:55:54.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f910159a700 1 -- 172.21.15.117:0/2633478874 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394861796 0 0) 0x7f91040b5510 con 0x7f9104143dd0 2020-11-20T13:55:54.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9101d9b700 1 --2- 172.21.15.117:0/2633478874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f910409b700 0x7f91041438e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f90f4002f30 tx=0x7f90f4006dc0).ready entity=mon.1 client_cookie=abeda1894c5c8a25 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f910159a700 1 -- 172.21.15.117:0/2633478874 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90f40097f0 con 0x7f910409b700 2020-11-20T13:55:54.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f910159a700 1 -- 172.21.15.117:0/2633478874 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4009950 con 0x7f910409b700 2020-11-20T13:55:54.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f910159a700 1 -- 172.21.15.117:0/2633478874 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90f4009cb0 con 0x7f910409b700 2020-11-20T13:55:54.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 -- 172.21.15.117:0/2633478874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f910409b700 msgr2=0x7f91041438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 --2- 172.21.15.117:0/2633478874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f910409b700 0x7f91041438e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f90f4002f30 tx=0x7f90f4006dc0).stop 2020-11-20T13:55:54.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 -- 172.21.15.117:0/2633478874 shutdown_connections 2020-11-20T13:55:54.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 --2- 172.21.15.117:0/2633478874 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9104098e30 0x7f910409b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 --2- 172.21.15.117:0/2633478874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f910409b700 0x7f91041438e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 -- 172.21.15.117:0/2633478874 >> 172.21.15.117:0/2633478874 conn(0x7f910405b9f0 msgr2=0x7f910405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 -- 172.21.15.117:0/2633478874 shutdown_connections 2020-11-20T13:55:54.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.189+0000 7f9108996700 1 -- 172.21.15.117:0/2633478874 wait complete. 2020-11-20T13:55:54.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.190+0000 7f9108996700 1 Processor -- start 2020-11-20T13:55:54.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.190+0000 7f9108996700 1 -- start start 2020-11-20T13:55:54.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9108996700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9104098e30 0x7f9104154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9108996700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f910409b700 0x7f9104149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9108996700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9104143dd0 0x7f9104139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9108996700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104155160 con 0x7f9104098e30 2020-11-20T13:55:54.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9108996700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104145520 con 0x7f9104143dd0 2020-11-20T13:55:54.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9108996700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9104154780 con 0x7f910409b700 2020-11-20T13:55:54.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9104143dd0 0x7f9104139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9104143dd0 0x7f9104139840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40190/0 (socket says 172.21.15.117:40190) 2020-11-20T13:55:54.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 -- 172.21.15.117:0/3278198411 learned_addr learned my addr 172.21.15.117:0/3278198411 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:54.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9101d9b700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f910409b700 0x7f9104149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f910259c700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9104098e30 0x7f9104154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 -- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f910409b700 msgr2=0x7f9104149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f910409b700 0x7f9104149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 -- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9104098e30 msgr2=0x7f9104154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9104098e30 0x7f9104154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 -- 172.21.15.117:0/3278198411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104154a20 con 0x7f9104143dd0 2020-11-20T13:55:54.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.191+0000 7f9102d9d700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9104143dd0 0x7f9104139840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f90fc000f00 tx=0x7f90fc007cd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.192+0000 7f90f37fe700 1 -- 172.21.15.117:0/3278198411 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90fc005b70 con 0x7f9104143dd0 2020-11-20T13:55:54.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.192+0000 7f90f37fe700 1 -- 172.21.15.117:0/3278198411 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc005cd0 con 0x7f9104143dd0 2020-11-20T13:55:54.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.192+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9104154cc0 con 0x7f9104143dd0 2020-11-20T13:55:54.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.192+0000 7f90f37fe700 1 -- 172.21.15.117:0/3278198411 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90fc007300 con 0x7f9104143dd0 2020-11-20T13:55:54.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.192+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91040afb40 con 0x7f9104143dd0 2020-11-20T13:55:54.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.193+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9104154e50 con 0x7f9104143dd0 2020-11-20T13:55:54.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.193+0000 7f90f37fe700 1 -- 172.21.15.117:0/3278198411 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f90fc010030 con 0x7f9104143dd0 2020-11-20T13:55:54.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.193+0000 7f90f37fe700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f90e0051320 0x7f90e00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.193+0000 7f90f37fe700 1 -- 172.21.15.117:0/3278198411 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f90fc069280 con 0x7f9104143dd0 2020-11-20T13:55:54.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.193+0000 7f910259c700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f90e0051320 0x7f90e00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.194+0000 7f910259c700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f90e0051320 0x7f90e00537e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f90f800c000 tx=0x7f90f8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.197+0000 7f90f37fe700 1 -- 172.21.15.117:0/3278198411 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f90fc041440 con 0x7f9104143dd0 2020-11-20T13:55:54.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.350+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9104057050 con 0x7f9104143dd0 2020-11-20T13:55:54.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.350+0000 7f90f37fe700 1 -- 172.21.15.117:0/3278198411 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f90fc044a60 con 0x7f9104143dd0 2020-11-20T13:55:54.351 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:54.351 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:54.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f90e0051320 msgr2=0x7f90e00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f90e0051320 0x7f90e00537e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f90f800c000 tx=0x7f90f8008040).stop 2020-11-20T13:55:54.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9104143dd0 msgr2=0x7f9104139840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9104143dd0 0x7f9104139840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f90fc000f00 tx=0x7f90fc007cd0).stop 2020-11-20T13:55:54.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 shutdown_connections 2020-11-20T13:55:54.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f90e0051320 0x7f90e00537e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9104143dd0 0x7f9104139840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9104098e30 0x7f9104154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 --2- 172.21.15.117:0/3278198411 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f910409b700 0x7f9104149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 >> 172.21.15.117:0/3278198411 conn(0x7f910405b9f0 msgr2=0x7f91040b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 shutdown_connections 2020-11-20T13:55:54.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.352+0000 7f9108996700 1 -- 172.21.15.117:0/3278198411 wait complete. 2020-11-20T13:55:54.363 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-20T13:55:54.363 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:55:54.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.468+0000 7fd25d9d0700 1 Processor -- start 2020-11-20T13:55:54.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.468+0000 7fd25d9d0700 1 -- start start 2020-11-20T13:55:54.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.469+0000 7fd25d9d0700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd258098e30 0x7fd25809b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.469+0000 7fd25d9d0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 0x7fd2581438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.469+0000 7fd25d9d0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2580b5510 con 0x7fd258143dd0 2020-11-20T13:55:54.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.469+0000 7fd25d9d0700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2580b57b0 con 0x7fd258098e30 2020-11-20T13:55:54.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.469+0000 7fd25d9d0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b5a50 con 0x7fd25809b700 2020-11-20T13:55:54.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd258143dd0 0x7fd2580b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59356/0 (socket says 172.21.15.117:59356) 2020-11-20T13:55:54.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 -- 172.21.15.117:0/1756959273 learned_addr learned my addr 172.21.15.117:0/1756959273 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:54.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2567fc700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 0x7fd2581438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd256ffd700 1 --2- 172.21.15.117:0/1756959273 >> v2:172.21.15.85:3300/0 conn(0x7fd258098e30 0x7fd25809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754611371 0 0) 0x7fd2580b5510 con 0x7fd258143dd0 2020-11-20T13:55:54.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd23c003230 con 0x7fd258143dd0 2020-11-20T13:55:54.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 106236972 0 0) 0x7fd23c003230 con 0x7fd258143dd0 2020-11-20T13:55:54.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 msgr2=0x7fd2581438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 0x7fd2581438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 >> v2:172.21.15.85:3300/0 conn(0x7fd258098e30 msgr2=0x7fd25809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 --2- 172.21.15.117:0/1756959273 >> v2:172.21.15.85:3300/0 conn(0x7fd258098e30 0x7fd25809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258099210 con 0x7fd258143dd0 2020-11-20T13:55:54.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fd250001a50 con 0x7fd258143dd0 2020-11-20T13:55:54.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 >> v1:172.21.15.117:6789/0 conn(0x7fd258143dd0 legacy=0x7fd2580b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 0x7fd23c003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd23c003c30 0x7fd23c0044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0049c0 0x7fd23c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b57b0 con 0x7fd23c003c30 2020-11-20T13:55:54.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b5a50 con 0x7fd23c0049c0 2020-11-20T13:55:54.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0041f0 con 0x7fd25809b700 2020-11-20T13:55:54.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0049c0 0x7fd23c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2567fc700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd23c003c30 0x7fd23c0044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd256ffd700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 0x7fd23c003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 -- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 msgr2=0x7fd23c003720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd25809b700 0x7fd23c003720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 -- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd23c003c30 msgr2=0x7fd23c0044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd23c003c30 0x7fd23c0044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.470+0000 7fd2577fe700 1 -- 172.21.15.117:0/1756959273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c003230 con 0x7fd23c0049c0 2020-11-20T13:55:54.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd2577fe700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0049c0 0x7fd23c004e40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fd250001760 tx=0x7fd250007ac0).ready entity=mon.1 client_cookie=47aa2ea9cd13899a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd250005b70 con 0x7fd23c0049c0 2020-11-20T13:55:54.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250005cd0 con 0x7fd23c0049c0 2020-11-20T13:55:54.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd255ffb700 1 -- 172.21.15.117:0/1756959273 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2500072c0 con 0x7fd23c0049c0 2020-11-20T13:55:54.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1756959273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0049c0 msgr2=0x7fd23c004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd25d9d0700 1 --2- 172.21.15.117:0/1756959273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0049c0 0x7fd23c004e40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fd250001760 tx=0x7fd250007ac0).stop 2020-11-20T13:55:54.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd2577fe700 1 -- 172.21.15.117:0/1756959273 reap_dead start 2020-11-20T13:55:54.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1756959273 shutdown_connections 2020-11-20T13:55:54.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1756959273 >> 172.21.15.117:0/1756959273 conn(0x7fd25805b9f0 msgr2=0x7fd25805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1756959273 shutdown_connections 2020-11-20T13:55:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.471+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1756959273 wait complete. 2020-11-20T13:55:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.472+0000 7fd25d9d0700 1 Processor -- start 2020-11-20T13:55:54.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.472+0000 7fd25d9d0700 1 -- start start 2020-11-20T13:55:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd25d9d0700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd2580b4560 0x7fd2580b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd25d9d0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd2580b2ea0 0x7fd258064cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd25d9d0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2580651b0 0x7fd258140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd25d9d0700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b0840 con 0x7fd2580b4560 2020-11-20T13:55:54.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd25d9d0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b05e0 con 0x7fd2580651b0 2020-11-20T13:55:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd25d9d0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd258145f60 con 0x7fd2580b2ea0 2020-11-20T13:55:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2580651b0 0x7fd258140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2580651b0 0x7fd258140c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40210/0 (socket says 172.21.15.117:40210) 2020-11-20T13:55:54.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 -- 172.21.15.117:0/1987066218 learned_addr learned my addr 172.21.15.117:0/1987066218 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:54.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2567fc700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd2580b2ea0 0x7fd258064cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd256ffd700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd2580b4560 0x7fd2580b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 -- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd2580b2ea0 msgr2=0x7fd258064cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd2580b2ea0 0x7fd258064cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 -- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd2580b4560 msgr2=0x7fd2580b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd2580b4560 0x7fd2580b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 -- 172.21.15.117:0/1987066218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258146200 con 0x7fd2580651b0 2020-11-20T13:55:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.473+0000 7fd2577fe700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2580651b0 0x7fd258140c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fd258153df0 tx=0x7fd250015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.474+0000 7fd247fff700 1 -- 172.21.15.117:0/1987066218 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd250013850 con 0x7fd2580651b0 2020-11-20T13:55:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.474+0000 7fd247fff700 1 -- 172.21.15.117:0/1987066218 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2500139b0 con 0x7fd2580651b0 2020-11-20T13:55:54.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.474+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2580b4940 con 0x7fd2580651b0 2020-11-20T13:55:54.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.474+0000 7fd247fff700 1 -- 172.21.15.117:0/1987066218 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd250013d10 con 0x7fd2580651b0 2020-11-20T13:55:54.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.474+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd258141340 con 0x7fd2580651b0 2020-11-20T13:55:54.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.474+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd258146390 con 0x7fd2580651b0 2020-11-20T13:55:54.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.475+0000 7fd247fff700 1 -- 172.21.15.117:0/1987066218 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd250026020 con 0x7fd2580651b0 2020-11-20T13:55:54.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.475+0000 7fd247fff700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2340513e0 0x7fd2340538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.475+0000 7fd247fff700 1 -- 172.21.15.117:0/1987066218 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd25006a630 con 0x7fd2580651b0 2020-11-20T13:55:54.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.475+0000 7fd256ffd700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2340513e0 0x7fd2340538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.475+0000 7fd256ffd700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2340513e0 0x7fd2340538a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd24c008000 tx=0x7fd24c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.478+0000 7fd247fff700 1 -- 172.21.15.117:0/1987066218 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd250042850 con 0x7fd2580651b0 2020-11-20T13:55:54.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.628+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd258057050 con 0x7fd2580651b0 2020-11-20T13:55:54.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.628+0000 7fd247fff700 1 -- 172.21.15.117:0/1987066218 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fd250045e70 con 0x7fd2580651b0 2020-11-20T13:55:54.629 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:54.629 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:55:54.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2340513e0 msgr2=0x7fd2340538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2340513e0 0x7fd2340538a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd24c008000 tx=0x7fd24c008040).stop 2020-11-20T13:55:54.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2580651b0 msgr2=0x7fd258140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2580651b0 0x7fd258140c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fd258153df0 tx=0x7fd250015000).stop 2020-11-20T13:55:54.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 shutdown_connections 2020-11-20T13:55:54.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2340513e0 0x7fd2340538a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2580651b0 0x7fd258140c10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd2580b4560 0x7fd2580b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 --2- 172.21.15.117:0/1987066218 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd2580b2ea0 0x7fd258064cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 >> 172.21.15.117:0/1987066218 conn(0x7fd25805b9f0 msgr2=0x7fd2580b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 shutdown_connections 2020-11-20T13:55:54.633 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.630+0000 7fd25d9d0700 1 -- 172.21.15.117:0/1987066218 wait complete. 2020-11-20T13:55:54.641 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-11-20T13:55:54.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.747+0000 7efc54da8700 1 Processor -- start 2020-11-20T13:55:54.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.747+0000 7efc54da8700 1 -- start start 2020-11-20T13:55:54.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc54da8700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7efc50147190 0x7efc50148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc54da8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc50061f40 0x7efc5014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc54da8700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc50147880 con 0x7efc5009aea0 2020-11-20T13:55:54.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc54da8700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc50147b20 con 0x7efc50147190 2020-11-20T13:55:54.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc54da8700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc50147dc0 con 0x7efc50061f40 2020-11-20T13:55:54.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc4ed9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc50061f40 0x7efc5014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc4dd9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7efc5009aea0 0x7efc500b8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59374/0 (socket says 172.21.15.117:59374) 2020-11-20T13:55:54.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc4dd9b700 1 -- 172.21.15.117:0/1004270875 learned_addr learned my addr 172.21.15.117:0/1004270875 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:54.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc4ed9d700 1 --2- 172.21.15.117:0/1004270875 >> v2:172.21.15.117:3300/0 conn(0x7efc50061f40 0x7efc5014acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40214/0 (socket says 172.21.15.117:40214) 2020-11-20T13:55:54.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.748+0000 7efc4e59c700 1 --2- 172.21.15.117:0/1004270875 >> v2:172.21.15.85:3300/0 conn(0x7efc50147190 0x7efc50148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4ed9d700 1 -- 172.21.15.117:0/1004270875 >> v2:172.21.15.85:3300/0 conn(0x7efc50147190 msgr2=0x7efc50148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4ed9d700 1 --2- 172.21.15.117:0/1004270875 >> v2:172.21.15.85:3300/0 conn(0x7efc50147190 0x7efc50148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4ed9d700 1 -- 172.21.15.117:0/1004270875 >> v1:172.21.15.117:6789/0 conn(0x7efc5009aea0 legacy=0x7efc500b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4ed9d700 1 -- 172.21.15.117:0/1004270875 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50147570 con 0x7efc50061f40 2020-11-20T13:55:54.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4d59a700 1 -- 172.21.15.117:0/1004270875 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841700648 0 0) 0x7efc50147880 con 0x7efc5009aea0 2020-11-20T13:55:54.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4e59c700 1 --2- 172.21.15.117:0/1004270875 >> v2:172.21.15.85:3300/0 conn(0x7efc50147190 0x7efc50148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:54.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4ed9d700 1 --2- 172.21.15.117:0/1004270875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc5014acb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efc480032b0 tx=0x7efc480091a0).ready entity=mon.1 client_cookie=4e10f4313ddcffd6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4d59a700 1 -- 172.21.15.117:0/1004270875 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4800ba50 con 0x7efc50061f40 2020-11-20T13:55:54.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4d59a700 1 -- 172.21.15.117:0/1004270875 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800bbb0 con 0x7efc50061f40 2020-11-20T13:55:54.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc4d59a700 1 -- 172.21.15.117:0/1004270875 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4800bd20 con 0x7efc50061f40 2020-11-20T13:55:54.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc54da8700 1 -- 172.21.15.117:0/1004270875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 msgr2=0x7efc5014acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.749+0000 7efc54da8700 1 --2- 172.21.15.117:0/1004270875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc5014acb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7efc480032b0 tx=0x7efc480091a0).stop 2020-11-20T13:55:54.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.750+0000 7efc54da8700 1 -- 172.21.15.117:0/1004270875 shutdown_connections 2020-11-20T13:55:54.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.750+0000 7efc54da8700 1 --2- 172.21.15.117:0/1004270875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc5014acb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.750+0000 7efc54da8700 1 --2- 172.21.15.117:0/1004270875 >> v2:172.21.15.85:3300/0 conn(0x7efc50147190 0x7efc50148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.750+0000 7efc54da8700 1 -- 172.21.15.117:0/1004270875 >> 172.21.15.117:0/1004270875 conn(0x7efc5005c4c0 msgr2=0x7efc500b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.750+0000 7efc54da8700 1 -- 172.21.15.117:0/1004270875 shutdown_connections 2020-11-20T13:55:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.750+0000 7efc54da8700 1 -- 172.21.15.117:0/1004270875 wait complete. 2020-11-20T13:55:54.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 Processor -- start 2020-11-20T13:55:54.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 -- start start 2020-11-20T13:55:54.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc500b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efc5009aea0 0x7efc5015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efc50147190 0x7efc5013db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501577a0 con 0x7efc5009aea0 2020-11-20T13:55:54.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5015aa70 con 0x7efc50061f40 2020-11-20T13:55:54.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc54da8700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc50144660 con 0x7efc50147190 2020-11-20T13:55:54.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc4e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc500b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc4e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc500b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40220/0 (socket says 172.21.15.117:40220) 2020-11-20T13:55:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc4e59c700 1 -- 172.21.15.117:0/2786223306 learned_addr learned my addr 172.21.15.117:0/2786223306 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc4dd9b700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efc5009aea0 0x7efc5015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.751+0000 7efc4ed9d700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efc50147190 0x7efc5013db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc4e59c700 1 -- 172.21.15.117:0/2786223306 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efc50147190 msgr2=0x7efc5013db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc4e59c700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7efc50147190 0x7efc5013db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc4e59c700 1 -- 172.21.15.117:0/2786223306 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efc5009aea0 msgr2=0x7efc5015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc4e59c700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7efc5009aea0 0x7efc5015a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc4e59c700 1 -- 172.21.15.117:0/2786223306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50144900 con 0x7efc50061f40 2020-11-20T13:55:54.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc4e59c700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc500b2da0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efc4400a000 tx=0x7efc44008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc44009980 con 0x7efc50061f40 2020-11-20T13:55:54.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44009ae0 con 0x7efc50061f40 2020-11-20T13:55:54.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc50144ba0 con 0x7efc50061f40 2020-11-20T13:55:54.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc44009e40 con 0x7efc50061f40 2020-11-20T13:55:54.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.752+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5009e050 con 0x7efc50061f40 2020-11-20T13:55:54.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc5013e470 con 0x7efc50061f40 2020-11-20T13:55:54.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7efc44013480 con 0x7efc50061f40 2020-11-20T13:55:54.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc3f7fe700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efc2c051320 0x7efc2c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7efc44069680 con 0x7efc50061f40 2020-11-20T13:55:54.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc4dd9b700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efc2c051320 0x7efc2c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc3f7fe700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6802/22905,v1:172.21.15.117:6803/22905] conn(0x7efc2c058e30 0x7efc2c05b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 --> [v2:172.21.15.117:6802/22905,v1:172.21.15.117:6803/22905] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc2c05b8a0 con 0x7efc2c058e30 2020-11-20T13:55:54.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc4406bcb0 con 0x7efc50061f40 2020-11-20T13:55:54.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc4ed9d700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6802/22905,v1:172.21.15.117:6803/22905] conn(0x7efc2c058e30 0x7efc2c05b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.753+0000 7efc4dd9b700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efc2c051320 0x7efc2c0537e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7efc4000d000 tx=0x7efc40007930).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.754+0000 7efc4ed9d700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6802/22905,v1:172.21.15.117:6803/22905] conn(0x7efc2c058e30 0x7efc2c05b250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.754+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== osd.0 v2:172.21.15.117:6802/22905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7efc2c05b8a0 con 0x7efc2c058e30 2020-11-20T13:55:54.773 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.771+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 --> [v2:172.21.15.117:6802/22905,v1:172.21.15.117:6803/22905] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efc500b9b10 con 0x7efc2c058e30 2020-11-20T13:55:54.773 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc3f7fe700 1 -- 172.21.15.117:0/2786223306 <== osd.0 v2:172.21.15.117:6802/22905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efc500b9b10 con 0x7efc2c058e30 2020-11-20T13:55:54.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6802/22905,v1:172.21.15.117:6803/22905] conn(0x7efc2c058e30 msgr2=0x7efc2c05b250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6802/22905,v1:172.21.15.117:6803/22905] conn(0x7efc2c058e30 0x7efc2c05b250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efc2c051320 msgr2=0x7efc2c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7efc2c051320 0x7efc2c0537e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7efc4000d000 tx=0x7efc40007930).stop 2020-11-20T13:55:54.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 msgr2=0x7efc500b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 --2- 172.21.15.117:0/2786223306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc50061f40 0x7efc500b2da0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efc4400a000 tx=0x7efc44008040).stop 2020-11-20T13:55:54.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc4ed9d700 1 -- 172.21.15.117:0/2786223306 reap_dead start 2020-11-20T13:55:54.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 shutdown_connections 2020-11-20T13:55:54.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 >> 172.21.15.117:0/2786223306 conn(0x7efc5005c4c0 msgr2=0x7efc500b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 shutdown_connections 2020-11-20T13:55:54.776 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.772+0000 7efc54da8700 1 -- 172.21.15.117:0/2786223306 wait complete. 2020-11-20T13:55:54.782 INFO:teuthology.orchestra.run.smithi117.stdout:55834574853 2020-11-20T13:55:54.783 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-11-20T13:55:54.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.890+0000 7f30dee91700 1 Processor -- start 2020-11-20T13:55:54.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.890+0000 7f30dee91700 1 -- start start 2020-11-20T13:55:54.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30dee91700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f30d81438e0 0x7f30d8143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30dee91700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d809b920 con 0x7f30d8098e30 2020-11-20T13:55:54.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30dee91700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d80b5510 con 0x7f30d80ae6f0 2020-11-20T13:55:54.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30dee91700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d8144230 con 0x7f30d81438e0 2020-11-20T13:55:54.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30dd42e700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f30d80ae6f0 0x7f30d80aeb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59388/0 (socket says 172.21.15.117:59388) 2020-11-20T13:55:54.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30dd42e700 1 -- 172.21.15.117:0/3638163646 learned_addr learned my addr 172.21.15.117:0/3638163646 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:54.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30d7fff700 1 --2- 172.21.15.117:0/3638163646 >> v2:172.21.15.85:3300/0 conn(0x7f30d81438e0 0x7f30d8143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102320621 0 0) 0x7f30d80b5510 con 0x7f30d80ae6f0 2020-11-20T13:55:54.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.891+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30c4003230 con 0x7f30d80ae6f0 2020-11-20T13:55:54.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2909528768 0 0) 0x7f30c4003230 con 0x7f30d80ae6f0 2020-11-20T13:55:54.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 >> v2:172.21.15.85:3300/0 conn(0x7f30d81438e0 msgr2=0x7f30d8143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 --2- 172.21.15.117:0/3638163646 >> v2:172.21.15.85:3300/0 conn(0x7f30d81438e0 0x7f30d8143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 >> v1:172.21.15.85:6789/0 conn(0x7f30d8098e30 legacy=0x7f30d809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d80b57b0 con 0x7f30d80ae6f0 2020-11-20T13:55:54.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f30d0001a50 con 0x7f30d80ae6f0 2020-11-20T13:55:54.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 >> v1:172.21.15.117:6789/0 conn(0x7f30d80ae6f0 legacy=0x7f30d80aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30c4003720 0x7f30c4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30c4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30c4004e40 0x7f30c400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8144230 con 0x7f30c4004e40 2020-11-20T13:55:54.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c4004340 con 0x7f30c40044d0 2020-11-20T13:55:54.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c400bc40 con 0x7f30c4003720 2020-11-20T13:55:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d7fff700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30c4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30dd42e700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30c4004e40 0x7f30c400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30dcc2d700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30c4003720 0x7f30c4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d7fff700 1 -- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30c4003720 msgr2=0x7f30c4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d7fff700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30c4003720 0x7f30c4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d7fff700 1 -- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30c4004e40 msgr2=0x7f30c400b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d7fff700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30c4004e40 0x7f30c400b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30dd42e700 1 -- 172.21.15.117:0/3638163646 reap_dead start 2020-11-20T13:55:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d7fff700 1 -- 172.21.15.117:0/3638163646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4003230 con 0x7f30c40044d0 2020-11-20T13:55:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d7fff700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30c4004950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f30cc00c000 tx=0x7f30cc008040).ready entity=mon.1 client_cookie=506d291d1b99127a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc00e8a0 con 0x7f30c40044d0 2020-11-20T13:55:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc00ea00 con 0x7f30c40044d0 2020-11-20T13:55:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30d77fe700 1 -- 172.21.15.117:0/3638163646 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30cc00ed60 con 0x7f30c40044d0 2020-11-20T13:55:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30dee91700 1 -- 172.21.15.117:0/3638163646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 msgr2=0x7f30c4004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.892+0000 7f30dee91700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30c4004950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f30cc00c000 tx=0x7f30cc008040).stop 2020-11-20T13:55:54.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.893+0000 7f30dee91700 1 -- 172.21.15.117:0/3638163646 shutdown_connections 2020-11-20T13:55:54.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.893+0000 7f30dee91700 1 --2- 172.21.15.117:0/3638163646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30c4004950 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.893+0000 7f30dee91700 1 -- 172.21.15.117:0/3638163646 >> 172.21.15.117:0/3638163646 conn(0x7f30d805b9f0 msgr2=0x7f30d805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.893+0000 7f30dee91700 1 -- 172.21.15.117:0/3638163646 shutdown_connections 2020-11-20T13:55:54.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.893+0000 7f30dee91700 1 -- 172.21.15.117:0/3638163646 wait complete. 2020-11-20T13:55:54.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.894+0000 7f30dee91700 1 Processor -- start 2020-11-20T13:55:54.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.894+0000 7f30dee91700 1 -- start start 2020-11-20T13:55:54.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30d8158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30d8146170 0x7f30d8154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30d8154e00 0x7f30d80b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8157910 con 0x7f30d8154e00 2020-11-20T13:55:54.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8157b40 con 0x7f30c40044d0 2020-11-20T13:55:54.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8157de0 con 0x7f30d8146170 2020-11-20T13:55:54.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30d8158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30d8158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40240/0 (socket says 172.21.15.117:40240) 2020-11-20T13:55:54.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 -- 172.21.15.117:0/1957786811 learned_addr learned my addr 172.21.15.117:0/1957786811 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:54.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30d7fff700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30d8146170 0x7f30d8154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dd42e700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30d8154e00 0x7f30d80b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 -- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30d8146170 msgr2=0x7f30d8154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30d8146170 0x7f30d8154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 -- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30d8154e00 msgr2=0x7f30d80b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30d8154e00 0x7f30d80b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 -- 172.21.15.117:0/1957786811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8158080 con 0x7f30c40044d0 2020-11-20T13:55:54.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30d7fff700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f30d8146170 0x7f30d8154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:54.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dd42e700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f30d8154e00 0x7f30d80b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:54.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dcc2d700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30d8158520 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f30d80b5040 tx=0x7f30c8007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30c8007020 con 0x7f30c40044d0 2020-11-20T13:55:54.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8007180 con 0x7f30c40044d0 2020-11-20T13:55:54.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d8155280 con 0x7f30c40044d0 2020-11-20T13:55:54.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30c8002600 con 0x7f30c40044d0 2020-11-20T13:55:54.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.895+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d80b0ec0 con 0x7f30c40044d0 2020-11-20T13:55:54.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.896+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f30d80b3d90 con 0x7f30c40044d0 2020-11-20T13:55:54.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.896+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f30c8002760 con 0x7f30c40044d0 2020-11-20T13:55:54.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.896+0000 7f30d5ffb700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f30b8051320 0x7f30b80537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f30c8068e80 con 0x7f30c40044d0 2020-11-20T13:55:54.914 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30d7fff700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f30b8051320 0x7f30b80537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.914 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30d5ffb700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6818/22907,v1:172.21.15.117:6819/22907] conn(0x7f30b8058e30 0x7f30b805b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:54.914 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 --> [v2:172.21.15.117:6818/22907,v1:172.21.15.117:6819/22907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30b805b8a0 con 0x7f30b8058e30 2020-11-20T13:55:54.914 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30c806b510 con 0x7f30c40044d0 2020-11-20T13:55:54.914 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30d7fff700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f30b8051320 0x7f30b80537e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f30cc008000 tx=0x7f30cc01a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30dd42e700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6818/22907,v1:172.21.15.117:6819/22907] conn(0x7f30b8058e30 0x7f30b805b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.897+0000 7f30dd42e700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6818/22907,v1:172.21.15.117:6819/22907] conn(0x7f30b8058e30 0x7f30b805b250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:54.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.898+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== osd.1 v2:172.21.15.117:6818/22907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f30b805b8a0 con 0x7f30b8058e30 2020-11-20T13:55:54.916 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.915+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 --> [v2:172.21.15.117:6818/22907,v1:172.21.15.117:6819/22907] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f30d80b65e0 con 0x7f30b8058e30 2020-11-20T13:55:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.915+0000 7f30d5ffb700 1 -- 172.21.15.117:0/1957786811 <== osd.1 v2:172.21.15.117:6818/22907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f30d80b65e0 con 0x7f30b8058e30 2020-11-20T13:55:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6818/22907,v1:172.21.15.117:6819/22907] conn(0x7f30b8058e30 msgr2=0x7f30b805b250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6818/22907,v1:172.21.15.117:6819/22907] conn(0x7f30b8058e30 0x7f30b805b250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f30b8051320 msgr2=0x7f30b80537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f30b8051320 0x7f30b80537e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f30cc008000 tx=0x7f30cc01a000).stop 2020-11-20T13:55:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 msgr2=0x7f30d8158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 --2- 172.21.15.117:0/1957786811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30c40044d0 0x7f30d8158520 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f30d80b5040 tx=0x7f30c8007ad0).stop 2020-11-20T13:55:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dd42e700 1 -- 172.21.15.117:0/1957786811 reap_dead start 2020-11-20T13:55:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 shutdown_connections 2020-11-20T13:55:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 >> 172.21.15.117:0/1957786811 conn(0x7f30d805b9f0 msgr2=0x7f30d809a860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 shutdown_connections 2020-11-20T13:55:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:54.916+0000 7f30dee91700 1 -- 172.21.15.117:0/1957786811 wait complete. 2020-11-20T13:55:54.926 INFO:teuthology.orchestra.run.smithi117.stdout:55834574852 2020-11-20T13:55:54.926 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-11-20T13:55:55.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.033+0000 7f0c63fff700 1 Processor -- start 2020-11-20T13:55:55.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.033+0000 7f0c63fff700 1 -- start start 2020-11-20T13:55:55.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c63fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c6409b6f0 0x7f0c64144860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c63fff700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6409c500 con 0x7f0c64098ab0 2020-11-20T13:55:55.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c63fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6409c7a0 con 0x7f0c640b9f90 2020-11-20T13:55:55.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c63fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6405fe00 con 0x7f0c6409b6f0 2020-11-20T13:55:55.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c62ffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c6409b6f0 0x7f0c64144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c637fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0c640b9f90 0x7f0c6405f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59414/0 (socket says 172.21.15.117:59414) 2020-11-20T13:55:55.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c62ffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c6409b6f0 0x7f0c64144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40250/0 (socket says 172.21.15.117:40250) 2020-11-20T13:55:55.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c637fe700 1 -- 172.21.15.117:0/4277211351 learned_addr learned my addr 172.21.15.117:0/4277211351 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:55.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c62ffd700 1 -- 172.21.15.117:0/4277211351 >> v1:172.21.15.117:6789/0 conn(0x7f0c640b9f90 legacy=0x7f0c6405f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c62ffd700 1 -- 172.21.15.117:0/4277211351 >> v1:172.21.15.85:6789/0 conn(0x7f0c64098ab0 legacy=0x7f0c6405ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c61ffb700 1 -- 172.21.15.117:0/4277211351 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402477824 0 0) 0x7f0c6409c7a0 con 0x7f0c640b9f90 2020-11-20T13:55:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c62ffd700 1 -- 172.21.15.117:0/4277211351 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6409ca40 con 0x7f0c6409b6f0 2020-11-20T13:55:55.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c62ffd700 1 --2- 172.21.15.117:0/4277211351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c6409b6f0 0x7f0c64144860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f0c640621c0 tx=0x7f0c54006dc0).ready entity=mon.1 client_cookie=35bdc6b12588bc96 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c61ffb700 1 -- 172.21.15.117:0/4277211351 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c540097b0 con 0x7f0c6409b6f0 2020-11-20T13:55:55.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c61ffb700 1 -- 172.21.15.117:0/4277211351 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54009910 con 0x7f0c6409b6f0 2020-11-20T13:55:55.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c61ffb700 1 -- 172.21.15.117:0/4277211351 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c54009c70 con 0x7f0c6409b6f0 2020-11-20T13:55:55.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c63fff700 1 -- 172.21.15.117:0/4277211351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c6409b6f0 msgr2=0x7f0c64144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.034+0000 7f0c63fff700 1 --2- 172.21.15.117:0/4277211351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c6409b6f0 0x7f0c64144860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f0c640621c0 tx=0x7f0c54006dc0).stop 2020-11-20T13:55:55.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.035+0000 7f0c63fff700 1 -- 172.21.15.117:0/4277211351 shutdown_connections 2020-11-20T13:55:55.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.035+0000 7f0c63fff700 1 --2- 172.21.15.117:0/4277211351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c6409b6f0 0x7f0c64144860 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.035+0000 7f0c63fff700 1 -- 172.21.15.117:0/4277211351 >> 172.21.15.117:0/4277211351 conn(0x7f0c640b9470 msgr2=0x7f0c640b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.035+0000 7f0c63fff700 1 -- 172.21.15.117:0/4277211351 shutdown_connections 2020-11-20T13:55:55.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.035+0000 7f0c63fff700 1 -- 172.21.15.117:0/4277211351 wait complete. 2020-11-20T13:55:55.047 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.036+0000 7f0c63fff700 1 Processor -- start 2020-11-20T13:55:55.047 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.036+0000 7f0c63fff700 1 -- start start 2020-11-20T13:55:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.036+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c64098ab0 0x7f0c6409e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c6409b6f0 0x7f0c641489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c640b9f90 0x7f0c64159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c63fff700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641566b0 con 0x7f0c64098ab0 2020-11-20T13:55:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c63fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6415a650 con 0x7f0c640b9f90 2020-11-20T13:55:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c63fff700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6415a8f0 con 0x7f0c6409b6f0 2020-11-20T13:55:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c640b9f90 0x7f0c64159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c640b9f90 0x7f0c64159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40260/0 (socket says 172.21.15.117:40260) 2020-11-20T13:55:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 -- 172.21.15.117:0/2229311711 learned_addr learned my addr 172.21.15.117:0/2229311711 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c62ffd700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c64098ab0 0x7f0c6409e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c627fc700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c6409b6f0 0x7f0c641489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 -- 172.21.15.117:0/2229311711 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c6409b6f0 msgr2=0x7f0c641489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0c6409b6f0 0x7f0c641489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 -- 172.21.15.117:0/2229311711 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c64098ab0 msgr2=0x7f0c6409e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0c64098ab0 0x7f0c6409e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 -- 172.21.15.117:0/2229311711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64148ed0 con 0x7f0c640b9f90 2020-11-20T13:55:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c637fe700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c640b9f90 0x7f0c64159cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0c5c000f00 tx=0x7f0c5c007cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c5c005960 con 0x7f0c640b9f90 2020-11-20T13:55:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c005ac0 con 0x7f0c640b9f90 2020-11-20T13:55:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c64067030 con 0x7f0c640b9f90 2020-11-20T13:55:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c5c005e20 con 0x7f0c640b9f90 2020-11-20T13:55:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.037+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6415a400 con 0x7f0c640b9f90 2020-11-20T13:55:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.038+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c64156800 con 0x7f0c640b9f90 2020-11-20T13:55:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.038+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0c5c010030 con 0x7f0c640b9f90 2020-11-20T13:55:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c53fff700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0c40051320 0x7f0c400537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0c5c068fa0 con 0x7f0c640b9f90 2020-11-20T13:55:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c62ffd700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0c40051320 0x7f0c400537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c53fff700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6810/22906,v1:172.21.15.117:6811/22906] conn(0x7f0c40058e30 0x7f0c4005b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 --> [v2:172.21.15.117:6810/22906,v1:172.21.15.117:6811/22906] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c4005b8a0 con 0x7f0c40058e30 2020-11-20T13:55:55.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c5c06b630 con 0x7f0c640b9f90 2020-11-20T13:55:55.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c627fc700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6810/22906,v1:172.21.15.117:6811/22906] conn(0x7f0c40058e30 0x7f0c4005b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c62ffd700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0c40051320 0x7f0c400537e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f0c5401b000 tx=0x7f0c54015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.039+0000 7f0c627fc700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6810/22906,v1:172.21.15.117:6811/22906] conn(0x7f0c40058e30 0x7f0c4005b250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.040+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== osd.2 v2:172.21.15.117:6810/22906 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f0c4005b8a0 con 0x7f0c40058e30 2020-11-20T13:55:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.057+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 --> [v2:172.21.15.117:6810/22906,v1:172.21.15.117:6811/22906] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c6415aaf0 con 0x7f0c40058e30 2020-11-20T13:55:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.057+0000 7f0c53fff700 1 -- 172.21.15.117:0/2229311711 <== osd.2 v2:172.21.15.117:6810/22906 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0c6415aaf0 con 0x7f0c40058e30 2020-11-20T13:55:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6810/22906,v1:172.21.15.117:6811/22906] conn(0x7f0c40058e30 msgr2=0x7f0c4005b250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6810/22906,v1:172.21.15.117:6811/22906] conn(0x7f0c40058e30 0x7f0c4005b250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0c40051320 msgr2=0x7f0c400537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0c40051320 0x7f0c400537e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f0c5401b000 tx=0x7f0c54015000).stop 2020-11-20T13:55:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c640b9f90 msgr2=0x7f0c64159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 --2- 172.21.15.117:0/2229311711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c640b9f90 0x7f0c64159cd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0c5c000f00 tx=0x7f0c5c007cb0).stop 2020-11-20T13:55:55.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c637fe700 1 -- 172.21.15.117:0/2229311711 reap_dead start 2020-11-20T13:55:55.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 shutdown_connections 2020-11-20T13:55:55.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 >> 172.21.15.117:0/2229311711 conn(0x7f0c640b9470 msgr2=0x7f0c640b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 shutdown_connections 2020-11-20T13:55:55.061 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.058+0000 7f0c63fff700 1 -- 172.21.15.117:0/2229311711 wait complete. 2020-11-20T13:55:55.068 INFO:teuthology.orchestra.run.smithi117.stdout:55834574852 2020-11-20T13:55:55.069 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2020-11-20T13:55:55.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.176+0000 7f8e5e6b2700 1 Processor -- start 2020-11-20T13:55:55.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.176+0000 7f8e5e6b2700 1 -- start start 2020-11-20T13:55:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.177+0000 7f8e5e6b2700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8e58098e30 0x7f8e5809b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.177+0000 7f8e5e6b2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e5809b700 0x7f8e581438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.177+0000 7f8e5e6b2700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e580b5510 con 0x7f8e58143dd0 2020-11-20T13:55:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.177+0000 7f8e5e6b2700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e580b57b0 con 0x7f8e58098e30 2020-11-20T13:55:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.177+0000 7f8e5e6b2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e580b5a50 con 0x7f8e5809b700 2020-11-20T13:55:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e57fff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8e58098e30 0x7f8e5809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e577fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e5809b700 0x7f8e581438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e57fff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8e58098e30 0x7f8e5809b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:51892/0 (socket says 172.21.15.117:51892) 2020-11-20T13:55:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e577fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e5809b700 0x7f8e581438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:40336/0 (socket says 172.21.15.117:40336) 2020-11-20T13:55:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e57fff700 1 -- 172.21.15.117:0/3893393092 learned_addr learned my addr 172.21.15.117:0/3893393092 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e577fe700 1 -- 172.21.15.117:0/3893393092 >> v2:172.21.15.85:3300/0 conn(0x7f8e58098e30 msgr2=0x7f8e5809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e577fe700 1 --2- 172.21.15.117:0/3893393092 >> v2:172.21.15.85:3300/0 conn(0x7f8e58098e30 0x7f8e5809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e577fe700 1 -- 172.21.15.117:0/3893393092 >> v1:172.21.15.85:6789/0 conn(0x7f8e58143dd0 legacy=0x7f8e580b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e577fe700 1 -- 172.21.15.117:0/3893393092 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58099210 con 0x7f8e5809b700 2020-11-20T13:55:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e57fff700 1 --2- 172.21.15.117:0/3893393092 >> v2:172.21.15.85:3300/0 conn(0x7f8e58098e30 0x7f8e5809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e577fe700 1 --2- 172.21.15.117:0/3893393092 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e5809b700 0x7f8e581438e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f8e4c002f30 tx=0x7f8e4c006dc0).ready entity=mon.2 client_cookie=c955d00e9c48eeb3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e56ffd700 1 -- 172.21.15.117:0/3893393092 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4c009760 con 0x7f8e5809b700 2020-11-20T13:55:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e56ffd700 1 -- 172.21.15.117:0/3893393092 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0098c0 con 0x7f8e5809b700 2020-11-20T13:55:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e56ffd700 1 -- 172.21.15.117:0/3893393092 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4c009c20 con 0x7f8e5809b700 2020-11-20T13:55:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/3893393092 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e5809b700 msgr2=0x7f8e581438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.178+0000 7f8e5e6b2700 1 --2- 172.21.15.117:0/3893393092 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e5809b700 0x7f8e581438e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f8e4c002f30 tx=0x7f8e4c006dc0).stop 2020-11-20T13:55:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.179+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/3893393092 shutdown_connections 2020-11-20T13:55:55.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.179+0000 7f8e5e6b2700 1 --2- 172.21.15.117:0/3893393092 >> v2:172.21.15.85:3300/0 conn(0x7f8e58098e30 0x7f8e5809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.179+0000 7f8e5e6b2700 1 --2- 172.21.15.117:0/3893393092 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e5809b700 0x7f8e581438e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.179+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/3893393092 >> 172.21.15.117:0/3893393092 conn(0x7f8e5805b9f0 msgr2=0x7f8e5805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.179+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/3893393092 shutdown_connections 2020-11-20T13:55:55.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.179+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/3893393092 wait complete. 2020-11-20T13:55:55.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.180+0000 7f8e5e6b2700 1 Processor -- start 2020-11-20T13:55:55.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.180+0000 7f8e5e6b2700 1 -- start start 2020-11-20T13:55:55.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.180+0000 7f8e5e6b2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e58098e30 0x7f8e5815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5e6b2700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8e5809b700 0x7f8e58155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5e6b2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58143dd0 0x7f8e58156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5e6b2700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5809e660 con 0x7f8e5809b700 2020-11-20T13:55:55.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5e6b2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e581567d0 con 0x7f8e58143dd0 2020-11-20T13:55:55.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5e6b2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e58156490 con 0x7f8e58098e30 2020-11-20T13:55:55.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58143dd0 0x7f8e58156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e57fff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e58098e30 0x7f8e5815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e57fff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e58098e30 0x7f8e5815a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:40338/0 (socket says 172.21.15.117:40338) 2020-11-20T13:55:55.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58143dd0 0x7f8e58156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40276/0 (socket says 172.21.15.117:40276) 2020-11-20T13:55:55.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e57fff700 1 -- 172.21.15.117:0/1337649288 learned_addr learned my addr 172.21.15.117:0/1337649288 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e577fe700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8e5809b700 0x7f8e58155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 -- 172.21.15.117:0/1337649288 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e58098e30 msgr2=0x7f8e5815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8e58098e30 0x7f8e5815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 -- 172.21.15.117:0/1337649288 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8e5809b700 msgr2=0x7f8e58155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8e5809b700 0x7f8e58155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 -- 172.21.15.117:0/1337649288 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5814a480 con 0x7f8e58143dd0 2020-11-20T13:55:55.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5cc4f700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58143dd0 0x7f8e58156c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8e58060b70 tx=0x7f8e50007df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e5000ce40 con 0x7f8e58143dd0 2020-11-20T13:55:55.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e50006860 con 0x7f8e58143dd0 2020-11-20T13:55:55.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5809e130 con 0x7f8e58143dd0 2020-11-20T13:55:55.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e50006bc0 con 0x7f8e58143dd0 2020-11-20T13:55:55.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.181+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e581573a0 con 0x7f8e58143dd0 2020-11-20T13:55:55.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.182+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e5814a610 con 0x7f8e58143dd0 2020-11-20T13:55:55.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.182+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8e50010070 con 0x7f8e58143dd0 2020-11-20T13:55:55.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e557fa700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8e380513c0 0x7f8e38053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8e50068530 con 0x7f8e58143dd0 2020-11-20T13:55:55.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e57fff700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8e380513c0 0x7f8e38053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e557fa700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x7f8e38058ed0 0x7f8e3805b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 --> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e3805b940 con 0x7f8e38058ed0 2020-11-20T13:55:55.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e5006ab60 con 0x7f8e58143dd0 2020-11-20T13:55:55.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e577fe700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x7f8e38058ed0 0x7f8e3805b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e57fff700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8e380513c0 0x7f8e38053880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e48008000 tx=0x7f8e48008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.183+0000 7f8e577fe700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x7f8e38058ed0 0x7f8e3805b2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.184+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== osd.3 v2:172.21.15.117:6826/22922 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f8e3805b940 con 0x7f8e38058ed0 2020-11-20T13:55:55.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.201+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 --> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e58156650 con 0x7f8e38058ed0 2020-11-20T13:55:55.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e557fa700 1 -- 172.21.15.117:0/1337649288 <== osd.3 v2:172.21.15.117:6826/22922 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e4c009e90 con 0x7f8e38058ed0 2020-11-20T13:55:55.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x7f8e38058ed0 msgr2=0x7f8e3805b2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x7f8e38058ed0 0x7f8e3805b2f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8e380513c0 msgr2=0x7f8e38053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8e380513c0 0x7f8e38053880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e48008000 tx=0x7f8e48008040).stop 2020-11-20T13:55:55.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58143dd0 msgr2=0x7f8e58156c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 --2- 172.21.15.117:0/1337649288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e58143dd0 0x7f8e58156c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8e58060b70 tx=0x7f8e50007df0).stop 2020-11-20T13:55:55.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5cc4f700 1 -- 172.21.15.117:0/1337649288 reap_dead start 2020-11-20T13:55:55.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 shutdown_connections 2020-11-20T13:55:55.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 >> 172.21.15.117:0/1337649288 conn(0x7f8e5805b9f0 msgr2=0x7f8e580b8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 shutdown_connections 2020-11-20T13:55:55.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.202+0000 7f8e5e6b2700 1 -- 172.21.15.117:0/1337649288 wait complete. 2020-11-20T13:55:55.213 INFO:teuthology.orchestra.run.smithi117.stdout:55834574854 2020-11-20T13:55:55.213 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2020-11-20T13:55:55.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.318+0000 7f027f88f700 1 Processor -- start 2020-11-20T13:55:55.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027f88f700 1 -- start start 2020-11-20T13:55:55.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027f88f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0278098e30 0x7f027809b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027f88f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f02780ae6f0 0x7f02780aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027f88f700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780b5510 con 0x7f027809b700 2020-11-20T13:55:55.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027f88f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02780b57b0 con 0x7f0278098e30 2020-11-20T13:55:55.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027f88f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b5a50 con 0x7f02780ae6f0 2020-11-20T13:55:55.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027d62b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0278098e30 0x7f027809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027d62b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0278098e30 0x7f027809b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40282/0 (socket says 172.21.15.117:40282) 2020-11-20T13:55:55.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.319+0000 7f027d62b700 1 -- 172.21.15.117:0/3733500032 learned_addr learned my addr 172.21.15.117:0/3733500032 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027de2c700 1 --2- 172.21.15.117:0/3733500032 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f02780ae6f0 0x7f02780aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027d62b700 1 -- 172.21.15.117:0/3733500032 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f02780ae6f0 msgr2=0x7f02780aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027d62b700 1 --2- 172.21.15.117:0/3733500032 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f02780ae6f0 0x7f02780aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027d62b700 1 -- 172.21.15.117:0/3733500032 >> v1:172.21.15.85:6789/0 conn(0x7f027809b700 legacy=0x7f02781438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027d62b700 1 -- 172.21.15.117:0/3733500032 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0278099210 con 0x7f0278098e30 2020-11-20T13:55:55.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027d62b700 1 --2- 172.21.15.117:0/3733500032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0278098e30 0x7f027809b210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0278060b70 tx=0x7f0268006e80).ready entity=mon.1 client_cookie=8a2449e580502615 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f0267fff700 1 -- 172.21.15.117:0/3733500032 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02680097b0 con 0x7f0278098e30 2020-11-20T13:55:55.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f0267fff700 1 -- 172.21.15.117:0/3733500032 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268009910 con 0x7f0278098e30 2020-11-20T13:55:55.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f0267fff700 1 -- 172.21.15.117:0/3733500032 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0268009ac0 con 0x7f0278098e30 2020-11-20T13:55:55.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027f88f700 1 -- 172.21.15.117:0/3733500032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0278098e30 msgr2=0x7f027809b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027f88f700 1 --2- 172.21.15.117:0/3733500032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0278098e30 0x7f027809b210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0278060b70 tx=0x7f0268006e80).stop 2020-11-20T13:55:55.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027f88f700 1 -- 172.21.15.117:0/3733500032 shutdown_connections 2020-11-20T13:55:55.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027f88f700 1 --2- 172.21.15.117:0/3733500032 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f02780ae6f0 0x7f02780aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027f88f700 1 --2- 172.21.15.117:0/3733500032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0278098e30 0x7f027809b210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.320+0000 7f027f88f700 1 -- 172.21.15.117:0/3733500032 >> 172.21.15.117:0/3733500032 conn(0x7f027805b9f0 msgr2=0x7f027805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.321+0000 7f027f88f700 1 -- 172.21.15.117:0/3733500032 shutdown_connections 2020-11-20T13:55:55.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.321+0000 7f027f88f700 1 -- 172.21.15.117:0/3733500032 wait complete. 2020-11-20T13:55:55.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.321+0000 7f027f88f700 1 Processor -- start 2020-11-20T13:55:55.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027f88f700 1 -- start start 2020-11-20T13:55:55.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027f88f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0278098e30 0x7f0278144f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027f88f700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f027809b700 0x7f0278146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027f88f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02780ae6f0 0x7f0278142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027f88f700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278148c80 con 0x7f027809b700 2020-11-20T13:55:55.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027f88f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278148f20 con 0x7f02780ae6f0 2020-11-20T13:55:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027f88f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02781491c0 con 0x7f0278098e30 2020-11-20T13:55:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027de2c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02780ae6f0 0x7f0278142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027de2c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02780ae6f0 0x7f0278142a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40292/0 (socket says 172.21.15.117:40292) 2020-11-20T13:55:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027de2c700 1 -- 172.21.15.117:0/3279772319 learned_addr learned my addr 172.21.15.117:0/3279772319 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027d62b700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0278098e30 0x7f0278144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.322+0000 7f027ce2a700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f027809b700 0x7f0278146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027de2c700 1 -- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0278098e30 msgr2=0x7f0278144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027de2c700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0278098e30 0x7f0278144f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027de2c700 1 -- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f027809b700 msgr2=0x7f0278146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027de2c700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f027809b700 0x7f0278146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027de2c700 1 -- 172.21.15.117:0/3279772319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027809d780 con 0x7f02780ae6f0 2020-11-20T13:55:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027ce2a700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f027809b700 0x7f0278146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027de2c700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02780ae6f0 0x7f0278142a30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0270007000 tx=0x7f0270008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02700078d0 con 0x7f02780ae6f0 2020-11-20T13:55:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270007a30 con 0x7f02780ae6f0 2020-11-20T13:55:55.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027809d910 con 0x7f02780ae6f0 2020-11-20T13:55:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0270007d90 con 0x7f02780ae6f0 2020-11-20T13:55:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.323+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781460f0 con 0x7f02780ae6f0 2020-11-20T13:55:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02781493c0 con 0x7f02780ae6f0 2020-11-20T13:55:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f027001b460 con 0x7f02780ae6f0 2020-11-20T13:55:55.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f02667fc700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0258051320 0x7f02580537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0270069410 con 0x7f02780ae6f0 2020-11-20T13:55:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f027d62b700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0258051320 0x7f02580537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f02667fc700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:6800/23126,v1:172.21.15.85:6801/23126] conn(0x7f0258058e30 0x7f025805b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 --> [v2:172.21.15.85:6800/23126,v1:172.21.15.85:6801/23126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f025805b8a0 con 0x7f0258058e30 2020-11-20T13:55:55.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.324+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f027006baa0 con 0x7f02780ae6f0 2020-11-20T13:55:55.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.325+0000 7f027d62b700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0258051320 0x7f02580537e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f0268019000 tx=0x7f0268016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.325+0000 7f027ce2a700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:6800/23126,v1:172.21.15.85:6801/23126] conn(0x7f0258058e30 0x7f025805b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.325+0000 7f027ce2a700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:6800/23126,v1:172.21.15.85:6801/23126] conn(0x7f0258058e30 0x7f025805b250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.326+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== osd.4 v2:172.21.15.85:6800/23126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f025805b8a0 con 0x7f0258058e30 2020-11-20T13:55:55.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.343+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 --> [v2:172.21.15.85:6800/23126,v1:172.21.15.85:6801/23126] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f02781431e0 con 0x7f0258058e30 2020-11-20T13:55:55.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f02667fc700 1 -- 172.21.15.117:0/3279772319 <== osd.4 v2:172.21.15.85:6800/23126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f02781431e0 con 0x7f0258058e30 2020-11-20T13:55:55.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:6800/23126,v1:172.21.15.85:6801/23126] conn(0x7f0258058e30 msgr2=0x7f025805b250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.85:6800/23126,v1:172.21.15.85:6801/23126] conn(0x7f0258058e30 0x7f025805b250 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0258051320 msgr2=0x7f02580537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f0258051320 0x7f02580537e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f0268019000 tx=0x7f0268016000).stop 2020-11-20T13:55:55.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02780ae6f0 msgr2=0x7f0278142a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 --2- 172.21.15.117:0/3279772319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02780ae6f0 0x7f0278142a30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0270007000 tx=0x7f0270008040).stop 2020-11-20T13:55:55.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027de2c700 1 -- 172.21.15.117:0/3279772319 reap_dead start 2020-11-20T13:55:55.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 shutdown_connections 2020-11-20T13:55:55.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 >> 172.21.15.117:0/3279772319 conn(0x7f027805b9f0 msgr2=0x7f02780b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 shutdown_connections 2020-11-20T13:55:55.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.344+0000 7f027f88f700 1 -- 172.21.15.117:0/3279772319 wait complete. 2020-11-20T13:55:55.355 INFO:teuthology.orchestra.run.smithi117.stdout:55834574855 2020-11-20T13:55:55.356 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2020-11-20T13:55:55.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.463+0000 7f47cada2700 1 Processor -- start 2020-11-20T13:55:55.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.463+0000 7f47cada2700 1 -- start start 2020-11-20T13:55:55.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.463+0000 7f47cada2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47c4098e30 0x7f47c409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47cada2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c409b700 0x7f47c41438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47cada2700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f47c4143dd0 0x7f47c40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47cada2700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c40b5510 con 0x7f47c4143dd0 2020-11-20T13:55:55.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47cada2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c40b57b0 con 0x7f47c4098e30 2020-11-20T13:55:55.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47cada2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40b5a50 con 0x7f47c409b700 2020-11-20T13:55:55.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47c4098e30 0x7f47c409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47c4098e30 0x7f47c409b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40298/0 (socket says 172.21.15.117:40298) 2020-11-20T13:55:55.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 -- 172.21.15.117:0/1005570667 learned_addr learned my addr 172.21.15.117:0/1005570667 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c3fff700 1 --2- 172.21.15.117:0/1005570667 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c409b700 0x7f47c41438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c933f700 1 --2- 172.21.15.117:0/1005570667 >> v2:172.21.15.85:3300/0 conn(0x7f47c4143dd0 0x7f47c40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 -- 172.21.15.117:0/1005570667 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c409b700 msgr2=0x7f47c41438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 --2- 172.21.15.117:0/1005570667 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c409b700 0x7f47c41438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 -- 172.21.15.117:0/1005570667 >> v2:172.21.15.85:3300/0 conn(0x7f47c4143dd0 msgr2=0x7f47c40b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 --2- 172.21.15.117:0/1005570667 >> v2:172.21.15.85:3300/0 conn(0x7f47c4143dd0 0x7f47c40b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 -- 172.21.15.117:0/1005570667 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4099210 con 0x7f47c4098e30 2020-11-20T13:55:55.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c8b3e700 1 --2- 172.21.15.117:0/1005570667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c4098e30 0x7f47c409b210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f47bc0032b0 tx=0x7f47bc009270).ready entity=mon.1 client_cookie=b27e4216b4536248 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c37fe700 1 -- 172.21.15.117:0/1005570667 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47bc00bc00 con 0x7f47c4098e30 2020-11-20T13:55:55.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c37fe700 1 -- 172.21.15.117:0/1005570667 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc00bd60 con 0x7f47c4098e30 2020-11-20T13:55:55.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47c37fe700 1 -- 172.21.15.117:0/1005570667 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47bc011400 con 0x7f47c4098e30 2020-11-20T13:55:55.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47cada2700 1 -- 172.21.15.117:0/1005570667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c4098e30 msgr2=0x7f47c409b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.464+0000 7f47cada2700 1 --2- 172.21.15.117:0/1005570667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c4098e30 0x7f47c409b210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f47bc0032b0 tx=0x7f47bc009270).stop 2020-11-20T13:55:55.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.465+0000 7f47cada2700 1 -- 172.21.15.117:0/1005570667 shutdown_connections 2020-11-20T13:55:55.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.465+0000 7f47cada2700 1 --2- 172.21.15.117:0/1005570667 >> v2:172.21.15.85:3300/0 conn(0x7f47c4143dd0 0x7f47c40b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.465+0000 7f47cada2700 1 --2- 172.21.15.117:0/1005570667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c4098e30 0x7f47c409b210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.465+0000 7f47cada2700 1 --2- 172.21.15.117:0/1005570667 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c409b700 0x7f47c41438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.465+0000 7f47cada2700 1 -- 172.21.15.117:0/1005570667 >> 172.21.15.117:0/1005570667 conn(0x7f47c405b9f0 msgr2=0x7f47c405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.465+0000 7f47cada2700 1 -- 172.21.15.117:0/1005570667 shutdown_connections 2020-11-20T13:55:55.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.465+0000 7f47cada2700 1 -- 172.21.15.117:0/1005570667 wait complete. 2020-11-20T13:55:55.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 Processor -- start 2020-11-20T13:55:55.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 -- start start 2020-11-20T13:55:55.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f47c4098e30 0x7f47c4134560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c409b700 0x7f47c4135690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c4143dd0 0x7f47c409c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c409e090 con 0x7f47c4098e30 2020-11-20T13:55:55.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4146bf0 con 0x7f47c409b700 2020-11-20T13:55:55.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.466+0000 7f47cada2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4146e90 con 0x7f47c4143dd0 2020-11-20T13:55:55.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c409b700 0x7f47c4135690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c409b700 0x7f47c4135690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40306/0 (socket says 172.21.15.117:40306) 2020-11-20T13:55:55.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 -- 172.21.15.117:0/11897666 learned_addr learned my addr 172.21.15.117:0/11897666 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c8b3e700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f47c4098e30 0x7f47c4134560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c933f700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c4143dd0 0x7f47c409c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 -- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c4143dd0 msgr2=0x7f47c409c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f47c4143dd0 0x7f47c409c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 -- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f47c4098e30 msgr2=0x7f47c4134560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f47c4098e30 0x7f47c4134560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 -- 172.21.15.117:0/11897666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4147130 con 0x7f47c409b700 2020-11-20T13:55:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c3fff700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c409b700 0x7f47c4135690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f47b4003070 tx=0x7f47b4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b4009790 con 0x7f47c409b700 2020-11-20T13:55:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b40098f0 con 0x7f47c409b700 2020-11-20T13:55:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c41447b0 con 0x7f47c409b700 2020-11-20T13:55:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47b4009c50 con 0x7f47c409b700 2020-11-20T13:55:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.467+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c409cc90 con 0x7f47c409b700 2020-11-20T13:55:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.468+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47c41472c0 con 0x7f47c409b700 2020-11-20T13:55:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.468+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f47b4013480 con 0x7f47c409b700 2020-11-20T13:55:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.468+0000 7f47c1ffb700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47a8051210 0x7f47a80536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f47b40697c0 con 0x7f47c409b700 2020-11-20T13:55:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c8b3e700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47a8051210 0x7f47a80536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c1ffb700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:6824/23124,v1:172.21.15.85:6825/23124] conn(0x7f47a8058ca0 0x7f47a805b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 --> [v2:172.21.15.85:6824/23124,v1:172.21.15.85:6825/23124] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47a805b710 con 0x7f47a8058ca0 2020-11-20T13:55:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b406e220 con 0x7f47c409b700 2020-11-20T13:55:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c8b3e700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47a8051210 0x7f47a80536d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f47bc01a000 tx=0x7f47bc016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c933f700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:6824/23124,v1:172.21.15.85:6825/23124] conn(0x7f47a8058ca0 0x7f47a805b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.469+0000 7f47c933f700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:6824/23124,v1:172.21.15.85:6825/23124] conn(0x7f47a8058ca0 0x7f47a805b0c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.470+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== osd.5 v2:172.21.15.85:6824/23124 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f47a805b710 con 0x7f47a8058ca0 2020-11-20T13:55:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.487+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 --> [v2:172.21.15.85:6824/23124,v1:172.21.15.85:6825/23124] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47c409e1d0 con 0x7f47a8058ca0 2020-11-20T13:55:55.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47c1ffb700 1 -- 172.21.15.117:0/11897666 <== osd.5 v2:172.21.15.85:6824/23124 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f47c409e1d0 con 0x7f47a8058ca0 2020-11-20T13:55:55.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:6824/23124,v1:172.21.15.85:6825/23124] conn(0x7f47a8058ca0 msgr2=0x7f47a805b0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.85:6824/23124,v1:172.21.15.85:6825/23124] conn(0x7f47a8058ca0 0x7f47a805b0c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47a8051210 msgr2=0x7f47a80536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47a8051210 0x7f47a80536d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f47bc01a000 tx=0x7f47bc016000).stop 2020-11-20T13:55:55.491 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c409b700 msgr2=0x7f47c4135690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.491 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 --2- 172.21.15.117:0/11897666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c409b700 0x7f47c4135690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f47b4003070 tx=0x7f47b4008040).stop 2020-11-20T13:55:55.491 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47c933f700 1 -- 172.21.15.117:0/11897666 reap_dead start 2020-11-20T13:55:55.491 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 shutdown_connections 2020-11-20T13:55:55.491 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 >> 172.21.15.117:0/11897666 conn(0x7f47c405b9f0 msgr2=0x7f47c40ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.491 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 shutdown_connections 2020-11-20T13:55:55.492 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.488+0000 7f47cada2700 1 -- 172.21.15.117:0/11897666 wait complete. 2020-11-20T13:55:55.499 INFO:teuthology.orchestra.run.smithi117.stdout:55834574853 2020-11-20T13:55:55.499 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2020-11-20T13:55:55.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.604+0000 7f46beb93700 1 Processor -- start 2020-11-20T13:55:55.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.604+0000 7f46beb93700 1 -- start start 2020-11-20T13:55:55.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46beb93700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8098e30 0x7f46b809b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46beb93700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f46b809b700 0x7f46b81438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46beb93700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b80b5510 con 0x7f46b8143dd0 2020-11-20T13:55:55.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46beb93700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b80b57b0 con 0x7f46b809b700 2020-11-20T13:55:55.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46beb93700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80b5a50 con 0x7f46b8098e30 2020-11-20T13:55:55.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46b7fff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f46b809b700 0x7f46b81438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46bc92f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8098e30 0x7f46b809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46bd130700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f46b8143dd0 0x7f46b80b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:43382/0 (socket says 172.21.15.117:43382) 2020-11-20T13:55:55.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.605+0000 7f46bd130700 1 -- 172.21.15.117:0/841768861 learned_addr learned my addr 172.21.15.117:0/841768861 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:55.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b7fff700 1 -- 172.21.15.117:0/841768861 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8098e30 msgr2=0x7f46b809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b7fff700 1 --2- 172.21.15.117:0/841768861 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8098e30 0x7f46b809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b7fff700 1 -- 172.21.15.117:0/841768861 >> v1:172.21.15.85:6789/0 conn(0x7f46b8143dd0 legacy=0x7f46b80b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b77fe700 1 -- 172.21.15.117:0/841768861 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780560967 0 0) 0x7f46b80b5510 con 0x7f46b8143dd0 2020-11-20T13:55:55.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b7fff700 1 -- 172.21.15.117:0/841768861 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8099210 con 0x7f46b809b700 2020-11-20T13:55:55.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b7fff700 1 --2- 172.21.15.117:0/841768861 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b809b700 0x7f46b81438e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f46ac002f30 tx=0x7f46ac006dc0).ready entity=mon.0 client_cookie=bc11d74b7e56ac6d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b77fe700 1 -- 172.21.15.117:0/841768861 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46ac009760 con 0x7f46b809b700 2020-11-20T13:55:55.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b77fe700 1 -- 172.21.15.117:0/841768861 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac0098c0 con 0x7f46b809b700 2020-11-20T13:55:55.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46b77fe700 1 -- 172.21.15.117:0/841768861 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46ac009c20 con 0x7f46b809b700 2020-11-20T13:55:55.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46beb93700 1 -- 172.21.15.117:0/841768861 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b809b700 msgr2=0x7f46b81438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.606+0000 7f46beb93700 1 --2- 172.21.15.117:0/841768861 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b809b700 0x7f46b81438e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f46ac002f30 tx=0x7f46ac006dc0).stop 2020-11-20T13:55:55.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.607+0000 7f46beb93700 1 -- 172.21.15.117:0/841768861 shutdown_connections 2020-11-20T13:55:55.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.607+0000 7f46beb93700 1 --2- 172.21.15.117:0/841768861 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8098e30 0x7f46b809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.607+0000 7f46beb93700 1 --2- 172.21.15.117:0/841768861 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b809b700 0x7f46b81438e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.607+0000 7f46beb93700 1 -- 172.21.15.117:0/841768861 >> 172.21.15.117:0/841768861 conn(0x7f46b805b9f0 msgr2=0x7f46b805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.607+0000 7f46beb93700 1 -- 172.21.15.117:0/841768861 shutdown_connections 2020-11-20T13:55:55.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.607+0000 7f46beb93700 1 -- 172.21.15.117:0/841768861 wait complete. 2020-11-20T13:55:55.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.607+0000 7f46beb93700 1 Processor -- start 2020-11-20T13:55:55.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46beb93700 1 -- start start 2020-11-20T13:55:55.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46beb93700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b8098e30 0x7f46b8157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46beb93700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b809b700 0x7f46b809ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46beb93700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8143dd0 0x7f46b814a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46beb93700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b8157e30 con 0x7f46b8098e30 2020-11-20T13:55:55.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46beb93700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b81580d0 con 0x7f46b809b700 2020-11-20T13:55:55.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46beb93700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b8158370 con 0x7f46b8143dd0 2020-11-20T13:55:55.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46b7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b809b700 0x7f46b809ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46b7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b809b700 0x7f46b809ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40322/0 (socket says 172.21.15.117:40322) 2020-11-20T13:55:55.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.608+0000 7f46b7fff700 1 -- 172.21.15.117:0/3907675359 learned_addr learned my addr 172.21.15.117:0/3907675359 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46bc92f700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b8098e30 0x7f46b8157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46bd130700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8143dd0 0x7f46b814a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b7fff700 1 -- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8143dd0 msgr2=0x7f46b814a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b7fff700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46b8143dd0 0x7f46b814a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b7fff700 1 -- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b8098e30 msgr2=0x7f46b8157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b7fff700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f46b8098e30 0x7f46b8157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b7fff700 1 -- 172.21.15.117:0/3907675359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8144250 con 0x7f46b809b700 2020-11-20T13:55:55.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b7fff700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b809b700 0x7f46b809ed30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f46ac002f30 tx=0x7f46ac015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46ac011ad0 con 0x7f46b809b700 2020-11-20T13:55:55.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac011c30 con 0x7f46b809b700 2020-11-20T13:55:55.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b8146990 con 0x7f46b809b700 2020-11-20T13:55:55.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46ac01d440 con 0x7f46b809b700 2020-11-20T13:55:55.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.609+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b814ad00 con 0x7f46b809b700 2020-11-20T13:55:55.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.610+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46b8157c80 con 0x7f46b809b700 2020-11-20T13:55:55.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.610+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f46ac024030 con 0x7f46b809b700 2020-11-20T13:55:55.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.610+0000 7f46b5ffb700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4698051210 0x7f46980536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.610+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f46ac06a0d0 con 0x7f46b809b700 2020-11-20T13:55:55.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.610+0000 7f46bc92f700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4698051210 0x7f46980536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.611+0000 7f46b5ffb700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:6815/23127,v1:172.21.15.85:6817/23127] conn(0x7f4698058d20 0x7f469805b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.611+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 --> [v2:172.21.15.85:6815/23127,v1:172.21.15.85:6817/23127] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f469805b790 con 0x7f4698058d20 2020-11-20T13:55:55.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.611+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ac06c760 con 0x7f46b809b700 2020-11-20T13:55:55.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.611+0000 7f46bc92f700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4698051210 0x7f46980536d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f46a800b000 tx=0x7f46a8006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.611+0000 7f46bd130700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:6815/23127,v1:172.21.15.85:6817/23127] conn(0x7f4698058d20 0x7f469805b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.611+0000 7f46bd130700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:6815/23127,v1:172.21.15.85:6817/23127] conn(0x7f4698058d20 0x7f469805b140 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.623 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.612+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== osd.6 v2:172.21.15.85:6815/23127 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f469805b790 con 0x7f4698058d20 2020-11-20T13:55:55.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.629+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 --> [v2:172.21.15.85:6815/23127,v1:172.21.15.85:6817/23127] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46b814b6b0 con 0x7f4698058d20 2020-11-20T13:55:55.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46b5ffb700 1 -- 172.21.15.117:0/3907675359 <== osd.6 v2:172.21.15.85:6815/23127 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f46b814b6b0 con 0x7f4698058d20 2020-11-20T13:55:55.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:6815/23127,v1:172.21.15.85:6817/23127] conn(0x7f4698058d20 msgr2=0x7f469805b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.85:6815/23127,v1:172.21.15.85:6817/23127] conn(0x7f4698058d20 0x7f469805b140 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4698051210 msgr2=0x7f46980536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.632 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4698051210 0x7f46980536d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f46a800b000 tx=0x7f46a8006dc0).stop 2020-11-20T13:55:55.633 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b809b700 msgr2=0x7f46b809ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.633 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 --2- 172.21.15.117:0/3907675359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b809b700 0x7f46b809ed30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f46ac002f30 tx=0x7f46ac015000).stop 2020-11-20T13:55:55.633 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46bd130700 1 -- 172.21.15.117:0/3907675359 reap_dead start 2020-11-20T13:55:55.633 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 shutdown_connections 2020-11-20T13:55:55.633 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 >> 172.21.15.117:0/3907675359 conn(0x7f46b805b9f0 msgr2=0x7f46b80b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.634 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 shutdown_connections 2020-11-20T13:55:55.634 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.630+0000 7f46beb93700 1 -- 172.21.15.117:0/3907675359 wait complete. 2020-11-20T13:55:55.641 INFO:teuthology.orchestra.run.smithi117.stdout:55834574854 2020-11-20T13:55:55.641 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2020-11-20T13:55:55.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.747+0000 7f48958ae700 1 Processor -- start 2020-11-20T13:55:55.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.747+0000 7f48958ae700 1 -- start start 2020-11-20T13:55:55.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f48958ae700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f4890098e30 0x7f489009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f48958ae700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900b5510 con 0x7f48900ae6f0 2020-11-20T13:55:55.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f48958ae700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900b57b0 con 0x7f489009b700 2020-11-20T13:55:55.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f48958ae700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48900b5a50 con 0x7f4890098e30 2020-11-20T13:55:55.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f488e7fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f489009b700 0x7f48901438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59490/0 (socket says 172.21.15.117:59490) 2020-11-20T13:55:55.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f488e7fc700 1 -- 172.21.15.117:0/1613491283 learned_addr learned my addr 172.21.15.117:0/1613491283 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:55.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f488effd700 1 --2- 172.21.15.117:0/1613491283 >> v2:172.21.15.85:3300/0 conn(0x7f4890098e30 0x7f489009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f488dffb700 1 -- 172.21.15.117:0/1613491283 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999095594 0 0) 0x7f48900b57b0 con 0x7f489009b700 2020-11-20T13:55:55.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.748+0000 7f488dffb700 1 -- 172.21.15.117:0/1613491283 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4874003230 con 0x7f489009b700 2020-11-20T13:55:55.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488effd700 1 -- 172.21.15.117:0/1613491283 >> v1:172.21.15.117:6789/0 conn(0x7f489009b700 legacy=0x7f48901438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488effd700 1 -- 172.21.15.117:0/1613491283 >> v1:172.21.15.85:6789/0 conn(0x7f48900ae6f0 legacy=0x7f48900aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488dffb700 1 -- 172.21.15.117:0/1613491283 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302673516 0 0) 0x7f48900b5510 con 0x7f48900ae6f0 2020-11-20T13:55:55.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488effd700 1 -- 172.21.15.117:0/1613491283 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890099210 con 0x7f4890098e30 2020-11-20T13:55:55.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488effd700 1 --2- 172.21.15.117:0/1613491283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 0x7f489009b210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f4884002f30 tx=0x7f4884006dc0).ready entity=mon.0 client_cookie=ae01371e6b9149fe server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488dffb700 1 -- 172.21.15.117:0/1613491283 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4884009800 con 0x7f4890098e30 2020-11-20T13:55:55.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488dffb700 1 -- 172.21.15.117:0/1613491283 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4884009960 con 0x7f4890098e30 2020-11-20T13:55:55.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f488dffb700 1 -- 172.21.15.117:0/1613491283 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4884009cc0 con 0x7f4890098e30 2020-11-20T13:55:55.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f48958ae700 1 -- 172.21.15.117:0/1613491283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 msgr2=0x7f489009b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f48958ae700 1 --2- 172.21.15.117:0/1613491283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 0x7f489009b210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f4884002f30 tx=0x7f4884006dc0).stop 2020-11-20T13:55:55.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f48958ae700 1 -- 172.21.15.117:0/1613491283 shutdown_connections 2020-11-20T13:55:55.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f48958ae700 1 --2- 172.21.15.117:0/1613491283 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 0x7f489009b210 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.749+0000 7f48958ae700 1 -- 172.21.15.117:0/1613491283 >> 172.21.15.117:0/1613491283 conn(0x7f489005b9f0 msgr2=0x7f489005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.750+0000 7f48958ae700 1 -- 172.21.15.117:0/1613491283 shutdown_connections 2020-11-20T13:55:55.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.750+0000 7f48958ae700 1 -- 172.21.15.117:0/1613491283 wait complete. 2020-11-20T13:55:55.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.750+0000 7f48958ae700 1 Processor -- start 2020-11-20T13:55:55.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.750+0000 7f48958ae700 1 -- start start 2020-11-20T13:55:55.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f48958ae700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 0x7f48900b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f48958ae700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f489009b700 0x7f48900b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f48958ae700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48900ae6f0 0x7f4890066ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f48958ae700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4890066fd0 con 0x7f4890098e30 2020-11-20T13:55:55.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f48958ae700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4890067260 con 0x7f48900ae6f0 2020-11-20T13:55:55.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f48958ae700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4890067500 con 0x7f489009b700 2020-11-20T13:55:55.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48900ae6f0 0x7f4890066ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48900ae6f0 0x7f4890066ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40340/0 (socket says 172.21.15.117:40340) 2020-11-20T13:55:55.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f488f7fe700 1 -- 172.21.15.117:0/3089013486 learned_addr learned my addr 172.21.15.117:0/3089013486 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f488effd700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 0x7f48900b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.751+0000 7f488e7fc700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f489009b700 0x7f48900b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f488f7fe700 1 -- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f489009b700 msgr2=0x7f48900b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f488f7fe700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f489009b700 0x7f48900b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f488f7fe700 1 -- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 msgr2=0x7f48900b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f488f7fe700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 0x7f48900b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f488f7fe700 1 -- 172.21.15.117:0/3089013486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890063400 con 0x7f48900ae6f0 2020-11-20T13:55:55.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f488effd700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4890098e30 0x7f48900b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:55.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f488f7fe700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48900ae6f0 0x7f4890066ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4888001210 tx=0x7f4888007d00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f488800a770 con 0x7f48900ae6f0 2020-11-20T13:55:55.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488800a8d0 con 0x7f48900ae6f0 2020-11-20T13:55:55.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4890155fd0 con 0x7f48900ae6f0 2020-11-20T13:55:55.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f488800ac30 con 0x7f48900ae6f0 2020-11-20T13:55:55.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.752+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4890142a20 con 0x7f48900ae6f0 2020-11-20T13:55:55.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4890057050 con 0x7f48900ae6f0 2020-11-20T13:55:55.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4888019030 con 0x7f48900ae6f0 2020-11-20T13:55:55.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f487ffff700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f486c051210 0x7f486c0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4888068aa0 con 0x7f48900ae6f0 2020-11-20T13:55:55.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f488effd700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f486c051210 0x7f486c0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f487ffff700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:6808/23128,v1:172.21.15.85:6809/23128] conn(0x7f486c058d20 0x7f486c05b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 --> [v2:172.21.15.85:6808/23128,v1:172.21.15.85:6809/23128] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f486c05b790 con 0x7f486c058d20 2020-11-20T13:55:55.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f488806d240 con 0x7f48900ae6f0 2020-11-20T13:55:55.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f488effd700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f486c051210 0x7f486c0536d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4884018000 tx=0x7f4884015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.753+0000 7f488e7fc700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:6808/23128,v1:172.21.15.85:6809/23128] conn(0x7f486c058d20 0x7f486c05b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.754+0000 7f488e7fc700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:6808/23128,v1:172.21.15.85:6809/23128] conn(0x7f486c058d20 0x7f486c05b140 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.755+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== osd.7 v2:172.21.15.85:6808/23128 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23260 (crc 0 0 0) 0x7f486c05b790 con 0x7f486c058d20 2020-11-20T13:55:55.773 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.772+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 --> [v2:172.21.15.85:6808/23128,v1:172.21.15.85:6809/23128] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4890156160 con 0x7f486c058d20 2020-11-20T13:55:55.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f487ffff700 1 -- 172.21.15.117:0/3089013486 <== osd.7 v2:172.21.15.85:6808/23128 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4890156160 con 0x7f486c058d20 2020-11-20T13:55:55.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:6808/23128,v1:172.21.15.85:6809/23128] conn(0x7f486c058d20 msgr2=0x7f486c05b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.85:6808/23128,v1:172.21.15.85:6809/23128] conn(0x7f486c058d20 0x7f486c05b140 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f486c051210 msgr2=0x7f486c0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.776 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f486c051210 0x7f486c0536d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4884018000 tx=0x7f4884015000).stop 2020-11-20T13:55:55.776 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48900ae6f0 msgr2=0x7f4890066ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.776 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 --2- 172.21.15.117:0/3089013486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48900ae6f0 0x7f4890066ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4888001210 tx=0x7f4888007d00).stop 2020-11-20T13:55:55.776 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f488f7fe700 1 -- 172.21.15.117:0/3089013486 reap_dead start 2020-11-20T13:55:55.776 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 shutdown_connections 2020-11-20T13:55:55.777 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 >> 172.21.15.117:0/3089013486 conn(0x7f489005b9f0 msgr2=0x7f48900b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.777 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 shutdown_connections 2020-11-20T13:55:55.777 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.773+0000 7f48958ae700 1 -- 172.21.15.117:0/3089013486 wait complete. 2020-11-20T13:55:55.784 INFO:teuthology.orchestra.run.smithi117.stdout:55834574854 2020-11-20T13:55:55.784 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-11-20T13:55:55.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.889+0000 7f9964b91700 1 Processor -- start 2020-11-20T13:55:55.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.889+0000 7f9964b91700 1 -- start start 2020-11-20T13:55:55.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f9964b91700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f9960098e30 0x7f996009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f9964b91700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f996009b700 0x7f99601438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f9964b91700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960143dd0 0x7f99600b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f9964b91700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99600b5510 con 0x7f9960098e30 2020-11-20T13:55:55.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f9964b91700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99600b57b0 con 0x7f996009b700 2020-11-20T13:55:55.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f9964b91700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99600b5a50 con 0x7f9960143dd0 2020-11-20T13:55:55.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f995dd9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f996009b700 0x7f99601438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f995dd9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f996009b700 0x7f99601438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40346/0 (socket says 172.21.15.117:40346) 2020-11-20T13:55:55.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.890+0000 7f995dd9b700 1 -- 172.21.15.117:0/19486635 learned_addr learned my addr 172.21.15.117:0/19486635 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995ed9d700 1 --2- 172.21.15.117:0/19486635 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960143dd0 0x7f99600b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995e59c700 1 --2- 172.21.15.117:0/19486635 >> v2:172.21.15.85:3300/0 conn(0x7f9960098e30 0x7f996009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995dd9b700 1 -- 172.21.15.117:0/19486635 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960143dd0 msgr2=0x7f99600b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995dd9b700 1 --2- 172.21.15.117:0/19486635 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960143dd0 0x7f99600b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995dd9b700 1 -- 172.21.15.117:0/19486635 >> v2:172.21.15.85:3300/0 conn(0x7f9960098e30 msgr2=0x7f996009b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995dd9b700 1 --2- 172.21.15.117:0/19486635 >> v2:172.21.15.85:3300/0 conn(0x7f9960098e30 0x7f996009b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995dd9b700 1 -- 172.21.15.117:0/19486635 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960099210 con 0x7f996009b700 2020-11-20T13:55:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995ed9d700 1 --2- 172.21.15.117:0/19486635 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960143dd0 0x7f99600b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:55.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995dd9b700 1 --2- 172.21.15.117:0/19486635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f99601438e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9950002f30 tx=0x7f9950006e80).ready entity=mon.1 client_cookie=89956673b89f108e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995d59a700 1 -- 172.21.15.117:0/19486635 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99500097f0 con 0x7f996009b700 2020-11-20T13:55:55.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995d59a700 1 -- 172.21.15.117:0/19486635 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950009950 con 0x7f996009b700 2020-11-20T13:55:55.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f995d59a700 1 -- 172.21.15.117:0/19486635 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9950009cb0 con 0x7f996009b700 2020-11-20T13:55:55.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f9964b91700 1 -- 172.21.15.117:0/19486635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 msgr2=0x7f99601438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.891+0000 7f9964b91700 1 --2- 172.21.15.117:0/19486635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f99601438e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9950002f30 tx=0x7f9950006e80).stop 2020-11-20T13:55:55.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.892+0000 7f9964b91700 1 -- 172.21.15.117:0/19486635 shutdown_connections 2020-11-20T13:55:55.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.892+0000 7f9964b91700 1 --2- 172.21.15.117:0/19486635 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960143dd0 0x7f99600b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.892+0000 7f9964b91700 1 --2- 172.21.15.117:0/19486635 >> v2:172.21.15.85:3300/0 conn(0x7f9960098e30 0x7f996009b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.892+0000 7f9964b91700 1 --2- 172.21.15.117:0/19486635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f99601438e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.892+0000 7f9964b91700 1 -- 172.21.15.117:0/19486635 >> 172.21.15.117:0/19486635 conn(0x7f996005b9f0 msgr2=0x7f996005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:55.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.892+0000 7f9964b91700 1 -- 172.21.15.117:0/19486635 shutdown_connections 2020-11-20T13:55:55.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.892+0000 7f9964b91700 1 -- 172.21.15.117:0/19486635 wait complete. 2020-11-20T13:55:55.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 Processor -- start 2020-11-20T13:55:55.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 -- start start 2020-11-20T13:55:55.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960098e30 0x7f996015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f9960066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9960143dd0 0x7f99600aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9960159a10 con 0x7f9960143dd0 2020-11-20T13:55:55.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9960159cb0 con 0x7f996009b700 2020-11-20T13:55:55.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f9964b91700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9960159f50 con 0x7f9960098e30 2020-11-20T13:55:55.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f9960066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f995dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f9960066f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40352/0 (socket says 172.21.15.117:40352) 2020-11-20T13:55:55.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f995dd9b700 1 -- 172.21.15.117:0/2655724229 learned_addr learned my addr 172.21.15.117:0/2655724229 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:55.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f995e59c700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960098e30 0x7f996015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.893+0000 7f995ed9d700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9960143dd0 0x7f99600aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f995dd9b700 1 -- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960098e30 msgr2=0x7f996015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f995dd9b700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960098e30 0x7f996015a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f995dd9b700 1 -- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9960143dd0 msgr2=0x7f99600aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:55.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f995dd9b700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9960143dd0 0x7f99600aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:55.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f995dd9b700 1 -- 172.21.15.117:0/2655724229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600af300 con 0x7f996009b700 2020-11-20T13:55:55.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f995dd9b700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f9960066f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9950002f30 tx=0x7f9950015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f994f7fe700 1 -- 172.21.15.117:0/2655724229 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9950011b40 con 0x7f996009b700 2020-11-20T13:55:55.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f994f7fe700 1 -- 172.21.15.117:0/2655724229 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950011ca0 con 0x7f996009b700 2020-11-20T13:55:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99601469f0 con 0x7f996009b700 2020-11-20T13:55:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f994f7fe700 1 -- 172.21.15.117:0/2655724229 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f995001e460 con 0x7f996009b700 2020-11-20T13:55:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.894+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9960140ca0 con 0x7f996009b700 2020-11-20T13:55:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.895+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9960159850 con 0x7f996009b700 2020-11-20T13:55:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.895+0000 7f994f7fe700 1 -- 172.21.15.117:0/2655724229 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9950025030 con 0x7f996009b700 2020-11-20T13:55:55.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.895+0000 7f994f7fe700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f993c051290 0x7f993c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:55.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.895+0000 7f994f7fe700 1 -- 172.21.15.117:0/2655724229 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f995006a3e0 con 0x7f996009b700 2020-11-20T13:55:55.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.895+0000 7f995e59c700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f993c051290 0x7f993c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:55.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.896+0000 7f995e59c700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f993c051290 0x7f993c053750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f995400c000 tx=0x7f9954008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:55.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:55.899+0000 7f994f7fe700 1 -- 172.21.15.117:0/2655724229 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9950042640 con 0x7f996009b700 2020-11-20T13:55:56.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.051+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f996015a110 con 0x7f996009b700 2020-11-20T13:55:56.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.051+0000 7f994f7fe700 1 -- 172.21.15.117:0/2655724229 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9950023030 con 0x7f996009b700 2020-11-20T13:55:56.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f993c051290 msgr2=0x7f993c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:56.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f993c051290 0x7f993c053750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f995400c000 tx=0x7f9954008040).stop 2020-11-20T13:55:56.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 msgr2=0x7f9960066f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:56.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f9960066f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9950002f30 tx=0x7f9950015000).stop 2020-11-20T13:55:56.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 shutdown_connections 2020-11-20T13:55:56.057 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f993c051290 0x7f993c053750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:56.057 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9960098e30 0x7f996015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:56.057 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9960143dd0 0x7f99600aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:56.057 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 --2- 172.21.15.117:0/2655724229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996009b700 0x7f9960066f20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:56.057 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 >> 172.21.15.117:0/2655724229 conn(0x7f996005b9f0 msgr2=0x7f99600af830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:56.058 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 shutdown_connections 2020-11-20T13:55:56.058 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:56.053+0000 7f9964b91700 1 -- 172.21.15.117:0/2655724229 wait complete. 2020-11-20T13:55:56.058 INFO:teuthology.orchestra.run.smithi117.stdout:55834574851 2020-11-20T13:55:56.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.0 2020-11-20T13:55:57.066 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-11-20T13:55:57.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.183+0000 7fa670850700 1 Processor -- start 2020-11-20T13:55:57.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.183+0000 7fa670850700 1 -- start start 2020-11-20T13:55:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa670850700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c09b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa670850700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fa66c09b700 0x7fa66c1438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa670850700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66c0b5510 con 0x7fa66c143dd0 2020-11-20T13:55:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa670850700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c0b57b0 con 0x7fa66c09b700 2020-11-20T13:55:57.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa670850700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0b5a50 con 0x7fa66c098e30 2020-11-20T13:55:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66a59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c09b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66a59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c09b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:40424/0 (socket says 172.21.15.117:40424) 2020-11-20T13:55:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66a59c700 1 -- 172.21.15.117:0/487768400 learned_addr learned my addr 172.21.15.117:0/487768400 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:57.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66ad9d700 1 --1- 172.21.15.117:0/487768400 >> v1:172.21.15.85:6789/0 conn(0x7fa66c143dd0 0x7fa66c0b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:43426/0 (socket says 172.21.15.117:43426) 2020-11-20T13:55:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa669d9b700 1 --2- 172.21.15.117:0/487768400 >> v2:172.21.15.85:3300/0 conn(0x7fa66c09b700 0x7fa66c1438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66a59c700 1 -- 172.21.15.117:0/487768400 >> v2:172.21.15.85:3300/0 conn(0x7fa66c09b700 msgr2=0x7fa66c1438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66a59c700 1 --2- 172.21.15.117:0/487768400 >> v2:172.21.15.85:3300/0 conn(0x7fa66c09b700 0x7fa66c1438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66a59c700 1 -- 172.21.15.117:0/487768400 >> v1:172.21.15.85:6789/0 conn(0x7fa66c143dd0 legacy=0x7fa66c0b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.184+0000 7fa66a59c700 1 -- 172.21.15.117:0/487768400 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c099210 con 0x7fa66c098e30 2020-11-20T13:55:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa66a59c700 1 --2- 172.21.15.117:0/487768400 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c09b210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa660002f30 tx=0x7fa660006e80).ready entity=mon.2 client_cookie=c05b85896fcd0a66 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa66959a700 1 -- 172.21.15.117:0/487768400 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6600097f0 con 0x7fa66c098e30 2020-11-20T13:55:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa66959a700 1 -- 172.21.15.117:0/487768400 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa660009950 con 0x7fa66c098e30 2020-11-20T13:55:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa66959a700 1 -- 172.21.15.117:0/487768400 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa660009cb0 con 0x7fa66c098e30 2020-11-20T13:55:57.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 -- 172.21.15.117:0/487768400 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 msgr2=0x7fa66c09b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 --2- 172.21.15.117:0/487768400 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c09b210 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa660002f30 tx=0x7fa660006e80).stop 2020-11-20T13:55:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 -- 172.21.15.117:0/487768400 shutdown_connections 2020-11-20T13:55:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 --2- 172.21.15.117:0/487768400 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c09b210 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 --2- 172.21.15.117:0/487768400 >> v2:172.21.15.85:3300/0 conn(0x7fa66c09b700 0x7fa66c1438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 -- 172.21.15.117:0/487768400 >> 172.21.15.117:0/487768400 conn(0x7fa66c05b9f0 msgr2=0x7fa66c05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:57.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 -- 172.21.15.117:0/487768400 shutdown_connections 2020-11-20T13:55:57.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.185+0000 7fa670850700 1 -- 172.21.15.117:0/487768400 wait complete. 2020-11-20T13:55:57.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.186+0000 7fa670850700 1 Processor -- start 2020-11-20T13:55:57.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.186+0000 7fa670850700 1 -- start start 2020-11-20T13:55:57.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.187+0000 7fa670850700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.187+0000 7fa670850700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c09b700 0x7fa66c09cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.187+0000 7fa670850700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa66c143dd0 0x7fa66c0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.187+0000 7fa670850700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0b2fa0 con 0x7fa66c143dd0 2020-11-20T13:55:57.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.187+0000 7fa670850700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c09e0f0 con 0x7fa66c09b700 2020-11-20T13:55:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.187+0000 7fa670850700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c09e320 con 0x7fa66c098e30 2020-11-20T13:55:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c09b700 0x7fa66c09cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c09b700 0x7fa66c09cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40366/0 (socket says 172.21.15.117:40366) 2020-11-20T13:55:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 -- 172.21.15.117:0/3183650143 learned_addr learned my addr 172.21.15.117:0/3183650143 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa66a59c700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa66ad9d700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa66c143dd0 0x7fa66c0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 -- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 msgr2=0x7fa66c138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 -- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa66c143dd0 msgr2=0x7fa66c0b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa66c143dd0 0x7fa66c0b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 -- 172.21.15.117:0/3183650143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c09d380 con 0x7fa66c09b700 2020-11-20T13:55:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa66a59c700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa669d9b700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c09b700 0x7fa66c09cb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa65c00c000 tx=0x7fa65c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa65b7fe700 1 -- 172.21.15.117:0/3183650143 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa65c00c820 con 0x7fa66c09b700 2020-11-20T13:55:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa65b7fe700 1 -- 172.21.15.117:0/3183650143 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c00c980 con 0x7fa66c09b700 2020-11-20T13:55:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c09d010 con 0x7fa66c09b700 2020-11-20T13:55:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa65b7fe700 1 -- 172.21.15.117:0/3183650143 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa65c00cce0 con 0x7fa66c09b700 2020-11-20T13:55:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.188+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c0b3520 con 0x7fa66c09b700 2020-11-20T13:55:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.189+0000 7fa65b7fe700 1 -- 172.21.15.117:0/3183650143 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa65c022070 con 0x7fa66c09b700 2020-11-20T13:55:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.189+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c09d510 con 0x7fa66c09b700 2020-11-20T13:55:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.190+0000 7fa65b7fe700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa648051290 0x7fa648053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.190+0000 7fa65b7fe700 1 -- 172.21.15.117:0/3183650143 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa65c069580 con 0x7fa66c09b700 2020-11-20T13:55:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.190+0000 7fa66a59c700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa648051290 0x7fa648053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.190+0000 7fa66a59c700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa648051290 0x7fa648053750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa66001c000 tx=0x7fa660015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.193+0000 7fa65b7fe700 1 -- 172.21.15.117:0/3183650143 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa65c041700 con 0x7fa66c09b700 2020-11-20T13:55:57.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.343+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa66c057050 con 0x7fa66c09b700 2020-11-20T13:55:57.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.344+0000 7fa65b7fe700 1 -- 172.21.15.117:0/3183650143 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa65c044d20 con 0x7fa66c09b700 2020-11-20T13:55:57.346 INFO:teuthology.orchestra.run.smithi117.stdout:55834574853 2020-11-20T13:55:57.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa648051290 msgr2=0x7fa648053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa648051290 0x7fa648053750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa66001c000 tx=0x7fa660015000).stop 2020-11-20T13:55:57.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c09b700 msgr2=0x7fa66c09cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c09b700 0x7fa66c09cb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fa65c00c000 tx=0x7fa65c008040).stop 2020-11-20T13:55:57.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 shutdown_connections 2020-11-20T13:55:57.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa648051290 0x7fa648053750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa66c098e30 0x7fa66c138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa66c143dd0 0x7fa66c0b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 --2- 172.21.15.117:0/3183650143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa66c09b700 0x7fa66c09cb20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 >> 172.21.15.117:0/3183650143 conn(0x7fa66c05b9f0 msgr2=0x7fa66c0ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:57.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 shutdown_connections 2020-11-20T13:55:57.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.345+0000 7fa670850700 1 -- 172.21.15.117:0/3183650143 wait complete. 2020-11-20T13:55:57.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2020-11-20T13:55:57.357 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-11-20T13:55:57.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.462+0000 7f341dad5700 1 Processor -- start 2020-11-20T13:55:57.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.462+0000 7f341dad5700 1 -- start start 2020-11-20T13:55:57.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f341dad5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 0x7f341809b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f341dad5700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f341809b700 0x7f34181438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f341dad5700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34180b5510 con 0x7f3418143dd0 2020-11-20T13:55:57.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f341dad5700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34180b57b0 con 0x7f341809b700 2020-11-20T13:55:57.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f341dad5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34180b5a50 con 0x7f3418098e30 2020-11-20T13:55:57.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f3417fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3418143dd0 0x7f34180b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59532/0 (socket says 172.21.15.117:59532) 2020-11-20T13:55:57.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f3417fff700 1 -- 172.21.15.117:0/3781802297 learned_addr learned my addr 172.21.15.117:0/3781802297 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:57.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3781802297 >> v2:172.21.15.85:3300/0 conn(0x7f341809b700 0x7f34181438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125913935 0 0) 0x7f34180b5510 con 0x7f3418143dd0 2020-11-20T13:55:57.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34177fe700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 0x7f341809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3404003230 con 0x7f3418143dd0 2020-11-20T13:55:57.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 948874743 0 0) 0x7f3404003230 con 0x7f3418143dd0 2020-11-20T13:55:57.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 msgr2=0x7f341809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:57.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 0x7f341809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 >> v2:172.21.15.85:3300/0 conn(0x7f341809b700 msgr2=0x7f34181438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:57.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 --2- 172.21.15.117:0/3781802297 >> v2:172.21.15.85:3300/0 conn(0x7f341809b700 0x7f34181438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418099210 con 0x7f3418143dd0 2020-11-20T13:55:57.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f3410001a50 con 0x7f3418143dd0 2020-11-20T13:55:57.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.463+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 >> v1:172.21.15.117:6789/0 conn(0x7f3418143dd0 legacy=0x7f34180b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3404003720 0x7f3404003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040044d0 0x7f3404004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 0x7f3404004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34180b57b0 con 0x7f3404003720 2020-11-20T13:55:57.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34180b5a50 con 0x7f34040044d0 2020-11-20T13:55:57.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3404004260 con 0x7f3418098e30 2020-11-20T13:55:57.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040044d0 0x7f3404004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34177fe700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3404003720 0x7f3404003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3417fff700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 0x7f3404004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3416ffd700 1 -- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 msgr2=0x7f3404004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:57.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3418098e30 0x7f3404004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3416ffd700 1 -- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3404003720 msgr2=0x7f3404003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:57.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3404003720 0x7f3404003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3416ffd700 1 -- 172.21.15.117:0/3781802297 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404003230 con 0x7f34040044d0 2020-11-20T13:55:57.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34177fe700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3404003720 0x7f3404003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:57.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040044d0 0x7f3404004950 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f340c00c000 tx=0x7f340c008040).ready entity=mon.1 client_cookie=c7b1701705a988f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c00e9d0 con 0x7f34040044d0 2020-11-20T13:55:57.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c00eb30 con 0x7f34040044d0 2020-11-20T13:55:57.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f34167fc700 1 -- 172.21.15.117:0/3781802297 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340c00ee90 con 0x7f34040044d0 2020-11-20T13:55:57.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f341dad5700 1 -- 172.21.15.117:0/3781802297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040044d0 msgr2=0x7f3404004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f341dad5700 1 --2- 172.21.15.117:0/3781802297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34040044d0 0x7f3404004950 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f340c00c000 tx=0x7f340c008040).stop 2020-11-20T13:55:57.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.464+0000 7f3417fff700 1 -- 172.21.15.117:0/3781802297 reap_dead start 2020-11-20T13:55:57.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.465+0000 7f341dad5700 1 -- 172.21.15.117:0/3781802297 shutdown_connections 2020-11-20T13:55:57.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.465+0000 7f341dad5700 1 -- 172.21.15.117:0/3781802297 >> 172.21.15.117:0/3781802297 conn(0x7f341805b9f0 msgr2=0x7f341805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:57.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.465+0000 7f341dad5700 1 -- 172.21.15.117:0/3781802297 shutdown_connections 2020-11-20T13:55:57.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.465+0000 7f341dad5700 1 -- 172.21.15.117:0/3781802297 wait complete. 2020-11-20T13:55:57.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 Processor -- start 2020-11-20T13:55:57.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 -- start start 2020-11-20T13:55:57.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3418138480 0x7f341813bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f341813c4b0 0x7f3418145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3418144b00 0x7f3418145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34181335d0 con 0x7f3418144b00 2020-11-20T13:55:57.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418146ad0 con 0x7f3418138480 2020-11-20T13:55:57.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f341dad5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3418146d70 con 0x7f341813c4b0 2020-11-20T13:55:57.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f34177fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3418138480 0x7f341813bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.466+0000 7f34177fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3418138480 0x7f341813bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40382/0 (socket says 172.21.15.117:40382) 2020-11-20T13:55:57.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f34177fe700 1 -- 172.21.15.117:0/3894170589 learned_addr learned my addr 172.21.15.117:0/3894170589 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:57.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f341813c4b0 0x7f3418145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f3417fff700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3418144b00 0x7f3418145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f34177fe700 1 -- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f341813c4b0 msgr2=0x7f3418145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f34177fe700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f341813c4b0 0x7f3418145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f34177fe700 1 -- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3418144b00 msgr2=0x7f3418145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f34177fe700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3418144b00 0x7f3418145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f34177fe700 1 -- 172.21.15.117:0/3894170589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418133250 con 0x7f3418138480 2020-11-20T13:55:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f34177fe700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3418138480 0x7f341813bfc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3408008000 tx=0x7f3408008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f3414ff9700 1 -- 172.21.15.117:0/3894170589 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340800e6f0 con 0x7f3418138480 2020-11-20T13:55:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f3414ff9700 1 -- 172.21.15.117:0/3894170589 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340800e850 con 0x7f3418138480 2020-11-20T13:55:57.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34181466a0 con 0x7f3418138480 2020-11-20T13:55:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f3414ff9700 1 -- 172.21.15.117:0/3894170589 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f340800ebb0 con 0x7f3418138480 2020-11-20T13:55:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.467+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3418145fb0 con 0x7f3418138480 2020-11-20T13:55:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.468+0000 7f3414ff9700 1 -- 172.21.15.117:0/3894170589 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3408024070 con 0x7f3418138480 2020-11-20T13:55:57.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.468+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34181333e0 con 0x7f3418138480 2020-11-20T13:55:57.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.468+0000 7f3414ff9700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f33f8051290 0x7f33f8053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.468+0000 7f3414ff9700 1 -- 172.21.15.117:0/3894170589 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3408069820 con 0x7f3418138480 2020-11-20T13:55:57.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.468+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f33f8051290 0x7f33f8053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.469+0000 7f3416ffd700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f33f8051290 0x7f33f8053750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f340c008000 tx=0x7f340c01a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.472+0000 7f3414ff9700 1 -- 172.21.15.117:0/3894170589 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3408041a40 con 0x7f3418138480 2020-11-20T13:55:57.624 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.622+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3418057050 con 0x7f3418138480 2020-11-20T13:55:57.624 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.623+0000 7f3414ff9700 1 -- 172.21.15.117:0/3894170589 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3408045060 con 0x7f3418138480 2020-11-20T13:55:57.624 INFO:teuthology.orchestra.run.smithi117.stdout:55834574853 2020-11-20T13:55:57.626 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f33f8051290 msgr2=0x7f33f8053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.626 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f33f8051290 0x7f33f8053750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f340c008000 tx=0x7f340c01a000).stop 2020-11-20T13:55:57.626 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3418138480 msgr2=0x7f341813bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.626 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3418138480 0x7f341813bfc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3408008000 tx=0x7f3408008040).stop 2020-11-20T13:55:57.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 shutdown_connections 2020-11-20T13:55:57.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f33f8051290 0x7f33f8053750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3418144b00 0x7f3418145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3418138480 0x7f341813bfc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 --2- 172.21.15.117:0/3894170589 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f341813c4b0 0x7f3418145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 >> 172.21.15.117:0/3894170589 conn(0x7f341805b9f0 msgr2=0x7f34180af280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:57.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 shutdown_connections 2020-11-20T13:55:57.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.624+0000 7f341dad5700 1 -- 172.21.15.117:0/3894170589 wait complete. 2020-11-20T13:55:57.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2020-11-20T13:55:57.636 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-11-20T13:55:57.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.743+0000 7fb5bc483700 1 Processor -- start 2020-11-20T13:55:57.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.743+0000 7fb5bc483700 1 -- start start 2020-11-20T13:55:57.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5bc483700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b4098e30 0x7fb5b409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5bc483700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb5b409b700 0x7fb5b41438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5bc483700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b40b5510 con 0x7fb5b4143dd0 2020-11-20T13:55:57.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5bc483700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b40b57b0 con 0x7fb5b409b700 2020-11-20T13:55:57.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5bc483700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40b5a50 con 0x7fb5b4098e30 2020-11-20T13:55:57.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5b9a1e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb5b409b700 0x7fb5b41438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5ba21f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b4098e30 0x7fb5b409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5baa20700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7fb5b4143dd0 0x7fb5b40b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:43454/0 (socket says 172.21.15.117:43454) 2020-11-20T13:55:57.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5b9a1e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb5b409b700 0x7fb5b41438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:52016/0 (socket says 172.21.15.117:52016) 2020-11-20T13:55:57.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5baa20700 1 -- 172.21.15.117:0/2932418760 learned_addr learned my addr 172.21.15.117:0/2932418760 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:57.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.744+0000 7fb5ba21f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b4098e30 0x7fb5b409b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:40456/0 (socket says 172.21.15.117:40456) 2020-11-20T13:55:57.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5ba21f700 1 -- 172.21.15.117:0/2932418760 >> v2:172.21.15.85:3300/0 conn(0x7fb5b409b700 msgr2=0x7fb5b41438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:57.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5ba21f700 1 --2- 172.21.15.117:0/2932418760 >> v2:172.21.15.85:3300/0 conn(0x7fb5b409b700 0x7fb5b41438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5ba21f700 1 -- 172.21.15.117:0/2932418760 >> v1:172.21.15.85:6789/0 conn(0x7fb5b4143dd0 legacy=0x7fb5b40b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5ba21f700 1 -- 172.21.15.117:0/2932418760 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b4099210 con 0x7fb5b4098e30 2020-11-20T13:55:57.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5b9a1e700 1 --2- 172.21.15.117:0/2932418760 >> v2:172.21.15.85:3300/0 conn(0x7fb5b409b700 0x7fb5b41438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:55:57.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5ba21f700 1 --2- 172.21.15.117:0/2932418760 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b4098e30 0x7fb5b409b210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb5a4002f30 tx=0x7fb5a4006dc0).ready entity=mon.2 client_cookie=4a93d615b54a35cf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5b921d700 1 -- 172.21.15.117:0/2932418760 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a4009760 con 0x7fb5b4098e30 2020-11-20T13:55:57.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5b921d700 1 -- 172.21.15.117:0/2932418760 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a40098c0 con 0x7fb5b4098e30 2020-11-20T13:55:57.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5b921d700 1 -- 172.21.15.117:0/2932418760 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a4009c20 con 0x7fb5b4098e30 2020-11-20T13:55:57.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5bc483700 1 -- 172.21.15.117:0/2932418760 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b4098e30 msgr2=0x7fb5b409b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.745+0000 7fb5bc483700 1 --2- 172.21.15.117:0/2932418760 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b4098e30 0x7fb5b409b210 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fb5a4002f30 tx=0x7fb5a4006dc0).stop 2020-11-20T13:55:57.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.746+0000 7fb5bc483700 1 -- 172.21.15.117:0/2932418760 shutdown_connections 2020-11-20T13:55:57.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.746+0000 7fb5bc483700 1 --2- 172.21.15.117:0/2932418760 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b4098e30 0x7fb5b409b210 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.746+0000 7fb5bc483700 1 --2- 172.21.15.117:0/2932418760 >> v2:172.21.15.85:3300/0 conn(0x7fb5b409b700 0x7fb5b41438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.746+0000 7fb5bc483700 1 -- 172.21.15.117:0/2932418760 >> 172.21.15.117:0/2932418760 conn(0x7fb5b405b9f0 msgr2=0x7fb5b405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:57.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.746+0000 7fb5bc483700 1 -- 172.21.15.117:0/2932418760 shutdown_connections 2020-11-20T13:55:57.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.746+0000 7fb5bc483700 1 -- 172.21.15.117:0/2932418760 wait complete. 2020-11-20T13:55:57.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 Processor -- start 2020-11-20T13:55:57.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 -- start start 2020-11-20T13:55:57.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b4098e30 0x7fb5b409f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b409b700 0x7fb5b4146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb5b4143dd0 0x7fb5b4138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b409d160 con 0x7fb5b4143dd0 2020-11-20T13:55:57.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b409d400 con 0x7fb5b4098e30 2020-11-20T13:55:57.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5bc483700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b412f190 con 0x7fb5b409b700 2020-11-20T13:55:57.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5ba21f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b4098e30 0x7fb5b409f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5ba21f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b4098e30 0x7fb5b409f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40396/0 (socket says 172.21.15.117:40396) 2020-11-20T13:55:57.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5ba21f700 1 -- 172.21.15.117:0/1393504877 learned_addr learned my addr 172.21.15.117:0/1393504877 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:57.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5b9a1e700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b409b700 0x7fb5b4146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.747+0000 7fb5baa20700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb5b4143dd0 0x7fb5b4138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5ba21f700 1 -- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b409b700 msgr2=0x7fb5b4146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5ba21f700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b409b700 0x7fb5b4146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5ba21f700 1 -- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb5b4143dd0 msgr2=0x7fb5b4138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5ba21f700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb5b4143dd0 0x7fb5b4138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5ba21f700 1 -- 172.21.15.117:0/1393504877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b412c5a0 con 0x7fb5b4098e30 2020-11-20T13:55:57.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5ba21f700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b4098e30 0x7fb5b409f450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fb5a4002f30 tx=0x7fb5a4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5a37fe700 1 -- 172.21.15.117:0/1393504877 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a40119e0 con 0x7fb5b4098e30 2020-11-20T13:55:57.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5a37fe700 1 -- 172.21.15.117:0/1393504877 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a4011b40 con 0x7fb5b4098e30 2020-11-20T13:55:57.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b412c730 con 0x7fb5b4098e30 2020-11-20T13:55:57.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5a37fe700 1 -- 172.21.15.117:0/1393504877 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb5a4011ea0 con 0x7fb5b4098e30 2020-11-20T13:55:57.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.748+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b409ce20 con 0x7fb5b4098e30 2020-11-20T13:55:57.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.749+0000 7fb5a37fe700 1 -- 172.21.15.117:0/1393504877 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb5a4024030 con 0x7fb5b4098e30 2020-11-20T13:55:57.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.749+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b409d5c0 con 0x7fb5b4098e30 2020-11-20T13:55:57.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.749+0000 7fb5a37fe700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb594051210 0x7fb5940536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:57.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.749+0000 7fb5a37fe700 1 -- 172.21.15.117:0/1393504877 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb5a4069d50 con 0x7fb5b4098e30 2020-11-20T13:55:57.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.749+0000 7fb5b9a1e700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb594051210 0x7fb5940536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:57.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.750+0000 7fb5b9a1e700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb594051210 0x7fb5940536d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb5b0008000 tx=0x7fb5b0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:57.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.753+0000 7fb5a37fe700 1 -- 172.21.15.117:0/1393504877 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb5a4045e60 con 0x7fb5b4098e30 2020-11-20T13:55:57.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.902+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb5b4057050 con 0x7fb5b4098e30 2020-11-20T13:55:57.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.902+0000 7fb5a37fe700 1 -- 172.21.15.117:0/1393504877 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb5a4022020 con 0x7fb5b4098e30 2020-11-20T13:55:57.904 INFO:teuthology.orchestra.run.smithi117.stdout:55834574853 2020-11-20T13:55:57.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb594051210 msgr2=0x7fb5940536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb594051210 0x7fb5940536d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb5b0008000 tx=0x7fb5b0008040).stop 2020-11-20T13:55:57.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b4098e30 msgr2=0x7fb5b409f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:57.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b4098e30 0x7fb5b409f450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fb5a4002f30 tx=0x7fb5a4015000).stop 2020-11-20T13:55:57.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 shutdown_connections 2020-11-20T13:55:57.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb594051210 0x7fb5940536d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb5b4143dd0 0x7fb5b4138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b4098e30 0x7fb5b409f450 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 --2- 172.21.15.117:0/1393504877 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb5b409b700 0x7fb5b4146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:57.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 >> 172.21.15.117:0/1393504877 conn(0x7fb5b405b9f0 msgr2=0x7fb5b40ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:57.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 shutdown_connections 2020-11-20T13:55:57.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:57.904+0000 7fb5bc483700 1 -- 172.21.15.117:0/1393504877 wait complete. 2020-11-20T13:55:57.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.2 2020-11-20T13:55:57.915 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2020-11-20T13:55:58.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.021+0000 7f5bf4902700 1 Processor -- start 2020-11-20T13:55:58.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.021+0000 7f5bf4902700 1 -- start start 2020-11-20T13:55:58.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bf4902700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5bf0098e30 0x7f5bf009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bf4902700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f5bf00ae6f0 0x7f5bf00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bf4902700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf00b5510 con 0x7f5bf009b700 2020-11-20T13:55:58.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bf4902700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf00b57b0 con 0x7f5bf00ae6f0 2020-11-20T13:55:58.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bf4902700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf00b5a50 con 0x7f5bf0098e30 2020-11-20T13:55:58.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bee59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5bf0098e30 0x7f5bf009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bedd9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f5bf009b700 0x7f5bf01438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59566/0 (socket says 172.21.15.117:59566) 2020-11-20T13:55:58.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bedd9b700 1 -- 172.21.15.117:0/114320616 learned_addr learned my addr 172.21.15.117:0/114320616 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:58.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5bee59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5bf0098e30 0x7f5bf009b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40404/0 (socket says 172.21.15.117:40404) 2020-11-20T13:55:58.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.022+0000 7f5beed9d700 1 --2- 172.21.15.117:0/114320616 >> v2:172.21.15.85:3300/0 conn(0x7f5bf00ae6f0 0x7f5bf00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bee59c700 1 -- 172.21.15.117:0/114320616 >> v2:172.21.15.85:3300/0 conn(0x7f5bf00ae6f0 msgr2=0x7f5bf00aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bee59c700 1 --2- 172.21.15.117:0/114320616 >> v2:172.21.15.85:3300/0 conn(0x7f5bf00ae6f0 0x7f5bf00aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bee59c700 1 -- 172.21.15.117:0/114320616 >> v1:172.21.15.117:6789/0 conn(0x7f5bf009b700 legacy=0x7f5bf01438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bed59a700 1 -- 172.21.15.117:0/114320616 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802105794 0 0) 0x7f5bf00b5510 con 0x7f5bf009b700 2020-11-20T13:55:58.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bee59c700 1 -- 172.21.15.117:0/114320616 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0099210 con 0x7f5bf0098e30 2020-11-20T13:55:58.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5beed9d700 1 --2- 172.21.15.117:0/114320616 >> v2:172.21.15.85:3300/0 conn(0x7f5bf00ae6f0 0x7f5bf00aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:58.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bee59c700 1 --2- 172.21.15.117:0/114320616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf009b210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f5be4002f30 tx=0x7f5be4006e50).ready entity=mon.1 client_cookie=74259814cfb27474 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bed59a700 1 -- 172.21.15.117:0/114320616 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be4010470 con 0x7f5bf0098e30 2020-11-20T13:55:58.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bed59a700 1 -- 172.21.15.117:0/114320616 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40105d0 con 0x7f5bf0098e30 2020-11-20T13:55:58.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bed59a700 1 -- 172.21.15.117:0/114320616 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be4010930 con 0x7f5bf0098e30 2020-11-20T13:55:58.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 -- 172.21.15.117:0/114320616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 msgr2=0x7f5bf009b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 --2- 172.21.15.117:0/114320616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf009b210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f5be4002f30 tx=0x7f5be4006e50).stop 2020-11-20T13:55:58.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 -- 172.21.15.117:0/114320616 shutdown_connections 2020-11-20T13:55:58.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 --2- 172.21.15.117:0/114320616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf009b210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 --2- 172.21.15.117:0/114320616 >> v2:172.21.15.85:3300/0 conn(0x7f5bf00ae6f0 0x7f5bf00aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 -- 172.21.15.117:0/114320616 >> 172.21.15.117:0/114320616 conn(0x7f5bf005b9f0 msgr2=0x7f5bf005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:58.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 -- 172.21.15.117:0/114320616 shutdown_connections 2020-11-20T13:55:58.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.023+0000 7f5bf4902700 1 -- 172.21.15.117:0/114320616 wait complete. 2020-11-20T13:55:58.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.024+0000 7f5bf4902700 1 Processor -- start 2020-11-20T13:55:58.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.024+0000 7f5bf4902700 1 -- start start 2020-11-20T13:55:58.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bf4902700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf0063400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bf4902700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5bf009b700 0x7f5bf0144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bf4902700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5bf00ae6f0 0x7f5bf0063c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bf4902700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf0152e60 con 0x7f5bf009b700 2020-11-20T13:55:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bf4902700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf00671d0 con 0x7f5bf0098e30 2020-11-20T13:55:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bf4902700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf0067470 con 0x7f5bf00ae6f0 2020-11-20T13:55:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf0063400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf0063400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40410/0 (socket says 172.21.15.117:40410) 2020-11-20T13:55:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 -- 172.21.15.117:0/1503330812 learned_addr learned my addr 172.21.15.117:0/1503330812 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 -- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5bf00ae6f0 msgr2=0x7f5bf0063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bedd9b700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5bf009b700 0x7f5bf0144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5beed9d700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5bf00ae6f0 0x7f5bf0063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5bf00ae6f0 0x7f5bf0063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 -- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5bf009b700 msgr2=0x7f5bf0144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5bf009b700 0x7f5bf0144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 -- 172.21.15.117:0/1503330812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0066d20 con 0x7f5bf0098e30 2020-11-20T13:55:58.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5beed9d700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5bf00ae6f0 0x7f5bf0063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:58.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bedd9b700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5bf009b700 0x7f5bf0144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:55:58.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.025+0000 7f5bee59c700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf0063400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5bf00b3320 tx=0x7f5be4006e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.026+0000 7f5bdf7fe700 1 -- 172.21.15.117:0/1503330812 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be4020070 con 0x7f5bf0098e30 2020-11-20T13:55:58.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.026+0000 7f5bdf7fe700 1 -- 172.21.15.117:0/1503330812 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40198b0 con 0x7f5bf0098e30 2020-11-20T13:55:58.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.026+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf0152980 con 0x7f5bf0098e30 2020-11-20T13:55:58.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.026+0000 7f5bdf7fe700 1 -- 172.21.15.117:0/1503330812 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5be4019c10 con 0x7f5bf0098e30 2020-11-20T13:55:58.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.026+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf0152c30 con 0x7f5bf0098e30 2020-11-20T13:55:58.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.027+0000 7f5bdf7fe700 1 -- 172.21.15.117:0/1503330812 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5be4019d70 con 0x7f5bf0098e30 2020-11-20T13:55:58.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.027+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf0057050 con 0x7f5bf0098e30 2020-11-20T13:55:58.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.027+0000 7f5bdf7fe700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5bcc051290 0x7f5bcc053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.027+0000 7f5bdf7fe700 1 -- 172.21.15.117:0/1503330812 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5be406ce60 con 0x7f5bf0098e30 2020-11-20T13:55:58.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.027+0000 7f5bedd9b700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5bcc051290 0x7f5bcc053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.027+0000 7f5bedd9b700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5bcc051290 0x7f5bcc053750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5be0008000 tx=0x7f5be0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.030+0000 7f5bdf7fe700 1 -- 172.21.15.117:0/1503330812 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5be4045080 con 0x7f5bf0098e30 2020-11-20T13:55:58.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.180+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5bf00b5250 con 0x7f5bf0098e30 2020-11-20T13:55:58.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.180+0000 7f5bdf7fe700 1 -- 172.21.15.117:0/1503330812 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5be40486a0 con 0x7f5bf0098e30 2020-11-20T13:55:58.182 INFO:teuthology.orchestra.run.smithi117.stdout:55834574854 2020-11-20T13:55:58.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.181+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5bcc051290 msgr2=0x7f5bcc053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.181+0000 7f5bf4902700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5bcc051290 0x7f5bcc053750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f5be0008000 tx=0x7f5be0009040).stop 2020-11-20T13:55:58.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.181+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 msgr2=0x7f5bf0063400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.181+0000 7f5bf4902700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf0063400 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f5bf00b3320 tx=0x7f5be4006e50).stop 2020-11-20T13:55:58.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 shutdown_connections 2020-11-20T13:55:58.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5bcc051290 0x7f5bcc053750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5bf00ae6f0 0x7f5bf0063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf0098e30 0x7f5bf0063400 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 --2- 172.21.15.117:0/1503330812 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5bf009b700 0x7f5bf0144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 >> 172.21.15.117:0/1503330812 conn(0x7f5bf005b9f0 msgr2=0x7f5bf00b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:58.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 shutdown_connections 2020-11-20T13:55:58.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.182+0000 7f5bf4902700 1 -- 172.21.15.117:0/1503330812 wait complete. 2020-11-20T13:55:58.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2020-11-20T13:55:58.194 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2020-11-20T13:55:58.302 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.300+0000 7fa50a2cc700 1 Processor -- start 2020-11-20T13:55:58.302 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.300+0000 7fa50a2cc700 1 -- start start 2020-11-20T13:55:58.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa50a2cc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa504098e30 0x7fa50409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa50a2cc700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5040b5510 con 0x7fa50409b700 2020-11-20T13:55:58.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa50a2cc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5040b57b0 con 0x7fa5040ae6f0 2020-11-20T13:55:58.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa50a2cc700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5040b5a50 con 0x7fa504098e30 2020-11-20T13:55:58.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa503fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa504098e30 0x7fa50409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa508869700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa5040ae6f0 0x7fa5040aeb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59582/0 (socket says 172.21.15.117:59582) 2020-11-20T13:55:58.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa508869700 1 -- 172.21.15.117:0/1275750735 learned_addr learned my addr 172.21.15.117:0/1275750735 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:58.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.301+0000 7fa503fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa504098e30 0x7fa50409b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40418/0 (socket says 172.21.15.117:40418) 2020-11-20T13:55:58.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa503fff700 1 -- 172.21.15.117:0/1275750735 >> v1:172.21.15.117:6789/0 conn(0x7fa5040ae6f0 legacy=0x7fa5040aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa503fff700 1 -- 172.21.15.117:0/1275750735 >> v1:172.21.15.85:6789/0 conn(0x7fa50409b700 legacy=0x7fa5041438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa503fff700 1 -- 172.21.15.117:0/1275750735 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa504099210 con 0x7fa504098e30 2020-11-20T13:55:58.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa503fff700 1 --2- 172.21.15.117:0/1275750735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504098e30 0x7fa50409b210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fa4f4002f30 tx=0x7fa4f4006dc0).ready entity=mon.1 client_cookie=b19186accbfe3857 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa502ffd700 1 -- 172.21.15.117:0/1275750735 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f40099e0 con 0x7fa504098e30 2020-11-20T13:55:58.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa502ffd700 1 -- 172.21.15.117:0/1275750735 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f4009b40 con 0x7fa504098e30 2020-11-20T13:55:58.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa502ffd700 1 -- 172.21.15.117:0/1275750735 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f4009ea0 con 0x7fa504098e30 2020-11-20T13:55:58.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa50a2cc700 1 -- 172.21.15.117:0/1275750735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504098e30 msgr2=0x7fa50409b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/1275750735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504098e30 0x7fa50409b210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fa4f4002f30 tx=0x7fa4f4006dc0).stop 2020-11-20T13:55:58.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa50a2cc700 1 -- 172.21.15.117:0/1275750735 shutdown_connections 2020-11-20T13:55:58.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/1275750735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa504098e30 0x7fa50409b210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa50a2cc700 1 -- 172.21.15.117:0/1275750735 >> 172.21.15.117:0/1275750735 conn(0x7fa50405b9f0 msgr2=0x7fa50405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:58.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa50a2cc700 1 -- 172.21.15.117:0/1275750735 shutdown_connections 2020-11-20T13:55:58.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.302+0000 7fa50a2cc700 1 -- 172.21.15.117:0/1275750735 wait complete. 2020-11-20T13:55:58.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.303+0000 7fa50a2cc700 1 Processor -- start 2020-11-20T13:55:58.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.303+0000 7fa50a2cc700 1 -- start start 2020-11-20T13:55:58.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa50a2cc700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa504098e30 0x7fa5040b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa50a2cc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50409b700 0x7fa5040b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa50a2cc700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa5040ae6f0 0x7fa504142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa50a2cc700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa504063860 con 0x7fa5040ae6f0 2020-11-20T13:55:58.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa50a2cc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa504063b00 con 0x7fa50409b700 2020-11-20T13:55:58.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa50a2cc700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041564a0 con 0x7fa504098e30 2020-11-20T13:55:58.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa5037fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50409b700 0x7fa5040b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa5037fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50409b700 0x7fa5040b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40426/0 (socket says 172.21.15.117:40426) 2020-11-20T13:55:58.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.304+0000 7fa5037fe700 1 -- 172.21.15.117:0/4017916561 learned_addr learned my addr 172.21.15.117:0/4017916561 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:58.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa508869700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa5040ae6f0 0x7fa504142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa503fff700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa504098e30 0x7fa5040b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5037fe700 1 -- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa504098e30 msgr2=0x7fa5040b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5037fe700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa504098e30 0x7fa5040b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5037fe700 1 -- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa5040ae6f0 msgr2=0x7fa504142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5037fe700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa5040ae6f0 0x7fa504142a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5037fe700 1 -- 172.21.15.117:0/4017916561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040b3040 con 0x7fa50409b700 2020-11-20T13:55:58.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5037fe700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50409b700 0x7fa5040b0920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa504060b70 tx=0x7fa4f8007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5017fa700 1 -- 172.21.15.117:0/4017916561 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f800e4a0 con 0x7fa50409b700 2020-11-20T13:55:58.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5017fa700 1 -- 172.21.15.117:0/4017916561 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f800e600 con 0x7fa50409b700 2020-11-20T13:55:58.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5040b0e10 con 0x7fa50409b700 2020-11-20T13:55:58.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa5017fa700 1 -- 172.21.15.117:0/4017916561 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f800e960 con 0x7fa50409b700 2020-11-20T13:55:58.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.305+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa504156240 con 0x7fa50409b700 2020-11-20T13:55:58.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.306+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5040b3230 con 0x7fa50409b700 2020-11-20T13:55:58.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.306+0000 7fa5017fa700 1 -- 172.21.15.117:0/4017916561 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa4f8019030 con 0x7fa50409b700 2020-11-20T13:55:58.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.306+0000 7fa5017fa700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa4e4051350 0x7fa4e4053810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.306+0000 7fa5017fa700 1 -- 172.21.15.117:0/4017916561 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa4f8068630 con 0x7fa50409b700 2020-11-20T13:55:58.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.306+0000 7fa503fff700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa4e4051350 0x7fa4e4053810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.307+0000 7fa503fff700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa4e4051350 0x7fa4e4053810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa4f4018000 tx=0x7fa4f4017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.310+0000 7fa5017fa700 1 -- 172.21.15.117:0/4017916561 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa4f80407b0 con 0x7fa50409b700 2020-11-20T13:55:58.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.462+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa504057050 con 0x7fa50409b700 2020-11-20T13:55:58.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.462+0000 7fa5017fa700 1 -- 172.21.15.117:0/4017916561 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa4f8007d40 con 0x7fa50409b700 2020-11-20T13:55:58.464 INFO:teuthology.orchestra.run.smithi117.stdout:55834574855 2020-11-20T13:55:58.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.463+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa4e4051350 msgr2=0x7fa4e4053810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.463+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa4e4051350 0x7fa4e4053810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa4f4018000 tx=0x7fa4f4017000).stop 2020-11-20T13:55:58.465 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.463+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50409b700 msgr2=0x7fa5040b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.463+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50409b700 0x7fa5040b0920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fa504060b70 tx=0x7fa4f8007c20).stop 2020-11-20T13:55:58.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 shutdown_connections 2020-11-20T13:55:58.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa4e4051350 0x7fa4e4053810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa504098e30 0x7fa5040b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.466 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa5040ae6f0 0x7fa504142a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 --2- 172.21.15.117:0/4017916561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa50409b700 0x7fa5040b0920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 >> 172.21.15.117:0/4017916561 conn(0x7fa50405b9f0 msgr2=0x7fa5040b6860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:58.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 shutdown_connections 2020-11-20T13:55:58.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.464+0000 7fa50a2cc700 1 -- 172.21.15.117:0/4017916561 wait complete. 2020-11-20T13:55:58.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2020-11-20T13:55:58.475 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2020-11-20T13:55:58.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.580+0000 7fc3d45c4700 1 Processor -- start 2020-11-20T13:55:58.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.580+0000 7fc3d45c4700 1 -- start start 2020-11-20T13:55:58.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d45c4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 0x7fc3cc146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d45c4700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc0b4a00 con 0x7fc3cc0b9d90 2020-11-20T13:55:58.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d45c4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc14af20 con 0x7fc3cc067b40 2020-11-20T13:55:58.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d45c4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc14b1c0 con 0x7fc3cc0be620 2020-11-20T13:55:58.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d1b5f700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc3cc067b40 0x7fc3cc0b8530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59592/0 (socket says 172.21.15.117:59592) 2020-11-20T13:55:58.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d1b5f700 1 -- 172.21.15.117:0/1924581543 learned_addr learned my addr 172.21.15.117:0/1924581543 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:58.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d2360700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 0x7fc3cc146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709607709 0 0) 0x7fc3cc14af20 con 0x7fc3cc067b40 2020-11-20T13:55:58.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.581+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b4003230 con 0x7fc3cc067b40 2020-11-20T13:55:58.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579494522 0 0) 0x7fc3cc0b4a00 con 0x7fc3cc0b9d90 2020-11-20T13:55:58.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3cc14af20 con 0x7fc3cc0b9d90 2020-11-20T13:55:58.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3818576357 0 0) 0x7fc3b4003230 con 0x7fc3cc067b40 2020-11-20T13:55:58.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 msgr2=0x7fc3cc146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 0x7fc3cc146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 >> v1:172.21.15.85:6789/0 conn(0x7fc3cc0b9d90 legacy=0x7fc3cc0b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc14b460 con 0x7fc3cc067b40 2020-11-20T13:55:58.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fc3c80016d0 con 0x7fc3cc067b40 2020-11-20T13:55:58.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 >> v1:172.21.15.117:6789/0 conn(0x7fc3cc067b40 legacy=0x7fc3cc0b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 0x7fc3b40038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b4003df0 0x7fc3b40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3b40049c0 0x7fc3b4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc14b1c0 con 0x7fc3b40049c0 2020-11-20T13:55:58.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4005390 con 0x7fc3b4003df0 2020-11-20T13:55:58.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b400b7a0 con 0x7fc3cc0be620 2020-11-20T13:55:58.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d1b5f700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b4003df0 0x7fc3b40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d2360700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 0x7fc3b40038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d2b61700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3b40049c0 0x7fc3b4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d1b5f700 1 -- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 msgr2=0x7fc3b40038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d1b5f700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc0be620 0x7fc3b40038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d1b5f700 1 -- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3b40049c0 msgr2=0x7fc3b4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d1b5f700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3b40049c0 0x7fc3b4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d1b5f700 1 -- 172.21.15.117:0/1924581543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b4003230 con 0x7fc3b4003df0 2020-11-20T13:55:58.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d2b61700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3b40049c0 0x7fc3b4004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:58.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d1b5f700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b4003df0 0x7fc3b40044d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fc3c80014a0 tx=0x7fc3cc0b3ea0).ready entity=mon.1 client_cookie=c4c911bb46352b65 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3c800cde0 con 0x7fc3b4003df0 2020-11-20T13:55:58.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80070d0 con 0x7fc3b4003df0 2020-11-20T13:55:58.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d135e700 1 -- 172.21.15.117:0/1924581543 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3c8007430 con 0x7fc3b4003df0 2020-11-20T13:55:58.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d45c4700 1 -- 172.21.15.117:0/1924581543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b4003df0 msgr2=0x7fc3b40044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d45c4700 1 --2- 172.21.15.117:0/1924581543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b4003df0 0x7fc3b40044d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fc3c80014a0 tx=0x7fc3cc0b3ea0).stop 2020-11-20T13:55:58.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.582+0000 7fc3d2b61700 1 -- 172.21.15.117:0/1924581543 reap_dead start 2020-11-20T13:55:58.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.583+0000 7fc3d45c4700 1 -- 172.21.15.117:0/1924581543 shutdown_connections 2020-11-20T13:55:58.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.583+0000 7fc3d45c4700 1 -- 172.21.15.117:0/1924581543 >> 172.21.15.117:0/1924581543 conn(0x7fc3cc05c4c0 msgr2=0x7fc3cc0bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:58.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.583+0000 7fc3d45c4700 1 -- 172.21.15.117:0/1924581543 shutdown_connections 2020-11-20T13:55:58.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.583+0000 7fc3d45c4700 1 -- 172.21.15.117:0/1924581543 wait complete. 2020-11-20T13:55:58.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.584+0000 7fc3d45c4700 1 Processor -- start 2020-11-20T13:55:58.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.584+0000 7fc3d45c4700 1 -- start start 2020-11-20T13:55:58.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.584+0000 7fc3d45c4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc0b6e70 0x7fc3cc156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.584+0000 7fc3d45c4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3cc0b7250 0x7fc3cc1439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d45c4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc143eb0 0x7fc3cc159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d45c4700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b48b0 con 0x7fc3cc0b7250 2020-11-20T13:55:58.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d45c4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc144320 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d45c4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc146ca0 con 0x7fc3cc143eb0 2020-11-20T13:55:58.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc0b6e70 0x7fc3cc156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc0b6e70 0x7fc3cc156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40442/0 (socket says 172.21.15.117:40442) 2020-11-20T13:55:58.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 -- 172.21.15.117:0/3339443083 learned_addr learned my addr 172.21.15.117:0/3339443083 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:58.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d1b5f700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3cc0b7250 0x7fc3cc1439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2b61700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc143eb0 0x7fc3cc159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 -- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc143eb0 msgr2=0x7fc3cc159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc143eb0 0x7fc3cc159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 -- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3cc0b7250 msgr2=0x7fc3cc1439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3cc0b7250 0x7fc3cc1439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 -- 172.21.15.117:0/3339443083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc146f40 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d2360700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc0b6e70 0x7fc3cc156de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fc3cc0b65c0 tx=0x7fc3bc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3bb7fe700 1 -- 172.21.15.117:0/3339443083 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3bc00fa30 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3bb7fe700 1 -- 172.21.15.117:0/3339443083 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc00fb90 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3bb7fe700 1 -- 172.21.15.117:0/3339443083 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3bc00fd00 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc1471e0 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.585+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc148470 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.586+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc147370 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.586+0000 7fc3bb7fe700 1 -- 172.21.15.117:0/3339443083 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc3bc024070 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.586+0000 7fc3bb7fe700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc3ac051210 0x7fc3ac0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.587+0000 7fc3bb7fe700 1 -- 172.21.15.117:0/3339443083 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc3bc069a30 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.587+0000 7fc3d1b5f700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc3ac051210 0x7fc3ac0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.587+0000 7fc3d1b5f700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc3ac051210 0x7fc3ac0536d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc3c801a000 tx=0x7fc3c8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.590+0000 7fc3bb7fe700 1 -- 172.21.15.117:0/3339443083 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc3bc041c50 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.742 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.741+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc3cc057050 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.741+0000 7fc3bb7fe700 1 -- 172.21.15.117:0/3339443083 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3bc045270 con 0x7fc3cc0b6e70 2020-11-20T13:55:58.743 INFO:teuthology.orchestra.run.smithi117.stdout:55834574853 2020-11-20T13:55:58.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc3ac051210 msgr2=0x7fc3ac0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc3ac051210 0x7fc3ac0536d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc3c801a000 tx=0x7fc3c8015000).stop 2020-11-20T13:55:58.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc0b6e70 msgr2=0x7fc3cc156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc0b6e70 0x7fc3cc156de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fc3cc0b65c0 tx=0x7fc3bc008040).stop 2020-11-20T13:55:58.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 shutdown_connections 2020-11-20T13:55:58.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc3ac051210 0x7fc3ac0536d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc3cc143eb0 0x7fc3cc159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc0b6e70 0x7fc3cc156de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 --2- 172.21.15.117:0/3339443083 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc3cc0b7250 0x7fc3cc1439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 >> 172.21.15.117:0/3339443083 conn(0x7fc3cc05c4c0 msgr2=0x7fc3cc0bd120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:58.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 shutdown_connections 2020-11-20T13:55:58.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.743+0000 7fc3d45c4700 1 -- 172.21.15.117:0/3339443083 wait complete. 2020-11-20T13:55:58.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2020-11-20T13:55:58.754 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-11-20T13:55:58.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.861+0000 7fa1a873c700 1 Processor -- start 2020-11-20T13:55:58.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.861+0000 7fa1a873c700 1 -- start start 2020-11-20T13:55:58.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.861+0000 7fa1a873c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 0x7fa1a009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.861+0000 7fa1a873c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fa1a00ae6f0 0x7fa1a00aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.861+0000 7fa1a873c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a00b5510 con 0x7fa1a009b700 2020-11-20T13:55:58.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.861+0000 7fa1a873c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a00b57b0 con 0x7fa1a00ae6f0 2020-11-20T13:55:58.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.861+0000 7fa1a873c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00b5a50 con 0x7fa1a0098e30 2020-11-20T13:55:58.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a5cd7700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa1a009b700 0x7fa1a01438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59612/0 (socket says 172.21.15.117:59612) 2020-11-20T13:55:58.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a5cd7700 1 -- 172.21.15.117:0/2818511554 learned_addr learned my addr 172.21.15.117:0/2818511554 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:58.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 0x7fa1a009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725008536 0 0) 0x7fa1a00b5510 con 0x7fa1a009b700 2020-11-20T13:55:58.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa188003230 con 0x7fa1a009b700 2020-11-20T13:55:58.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a6cd9700 1 --2- 172.21.15.117:0/2818511554 >> v2:172.21.15.85:3300/0 conn(0x7fa1a00ae6f0 0x7fa1a00aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 711224900 0 0) 0x7fa188003230 con 0x7fa1a009b700 2020-11-20T13:55:58.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 msgr2=0x7fa1a009b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 0x7fa1a009b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 >> v2:172.21.15.85:3300/0 conn(0x7fa1a00ae6f0 msgr2=0x7fa1a00aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 --2- 172.21.15.117:0/2818511554 >> v2:172.21.15.85:3300/0 conn(0x7fa1a00ae6f0 0x7fa1a00aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a0099210 con 0x7fa1a009b700 2020-11-20T13:55:58.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fa19c0016d0 con 0x7fa1a009b700 2020-11-20T13:55:58.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a6cd9700 1 --2- 172.21.15.117:0/2818511554 >> v2:172.21.15.85:3300/0 conn(0x7fa1a00ae6f0 0x7fa1a00aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:58.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 >> v1:172.21.15.117:6789/0 conn(0x7fa1a009b700 legacy=0x7fa1a01438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa188003720 0x7fa188003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1880044d0 0x7fa188004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 0x7fa188004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00b57b0 con 0x7fa1880044d0 2020-11-20T13:55:58.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00b5a50 con 0x7fa188003720 2020-11-20T13:55:58.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa188004260 con 0x7fa1a0098e30 2020-11-20T13:55:58.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa188003720 0x7fa188003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a6cd9700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 0x7fa188004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a5cd7700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1880044d0 0x7fa188004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a64d8700 1 -- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 msgr2=0x7fa188004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a0098e30 0x7fa188004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a64d8700 1 -- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1880044d0 msgr2=0x7fa188004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:58.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1880044d0 0x7fa188004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.862+0000 7fa1a64d8700 1 -- 172.21.15.117:0/2818511554 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188003230 con 0x7fa188003720 2020-11-20T13:55:58.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa188003720 0x7fa188003b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fa19000c000 tx=0x7fa190008040).ready entity=mon.1 client_cookie=8d46b10a8309e8fe server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa19000e9d0 con 0x7fa188003720 2020-11-20T13:55:58.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19000eb30 con 0x7fa188003720 2020-11-20T13:55:58.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a54d6700 1 -- 172.21.15.117:0/2818511554 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa19000ee90 con 0x7fa188003720 2020-11-20T13:55:58.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a873c700 1 -- 172.21.15.117:0/2818511554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa188003720 msgr2=0x7fa188003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a873c700 1 --2- 172.21.15.117:0/2818511554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa188003720 0x7fa188003b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fa19000c000 tx=0x7fa190008040).stop 2020-11-20T13:55:58.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a6cd9700 1 -- 172.21.15.117:0/2818511554 reap_dead start 2020-11-20T13:55:58.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a873c700 1 -- 172.21.15.117:0/2818511554 shutdown_connections 2020-11-20T13:55:58.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a873c700 1 -- 172.21.15.117:0/2818511554 >> 172.21.15.117:0/2818511554 conn(0x7fa1a005b9f0 msgr2=0x7fa1a005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:58.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a873c700 1 -- 172.21.15.117:0/2818511554 shutdown_connections 2020-11-20T13:55:58.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.863+0000 7fa1a873c700 1 -- 172.21.15.117:0/2818511554 wait complete. 2020-11-20T13:55:58.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.864+0000 7fa1a873c700 1 Processor -- start 2020-11-20T13:55:58.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.864+0000 7fa1a873c700 1 -- start start 2020-11-20T13:55:58.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a873c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a00b27b0 0x7fa1a013ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a873c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a00b3060 0x7fa1a0131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a873c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1a0131d10 0x7fa1a00b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a873c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a0132190 con 0x7fa1a0131d10 2020-11-20T13:55:58.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a873c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a0151ab0 con 0x7fa1a00b3060 2020-11-20T13:55:58.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a873c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a013b420 con 0x7fa1a00b27b0 2020-11-20T13:55:58.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a00b3060 0x7fa1a0131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a00b3060 0x7fa1a0131820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40460/0 (socket says 172.21.15.117:40460) 2020-11-20T13:55:58.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 -- 172.21.15.117:0/1152154968 learned_addr learned my addr 172.21.15.117:0/1152154968 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:58.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a6cd9700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1a0131d10 0x7fa1a00b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a00b27b0 0x7fa1a013ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 -- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a00b27b0 msgr2=0x7fa1a013ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a00b27b0 0x7fa1a013ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 -- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1a0131d10 msgr2=0x7fa1a00b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:58.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1a0131d10 0x7fa1a00b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:58.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 -- 172.21.15.117:0/1152154968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a013b6c0 con 0x7fa1a00b3060 2020-11-20T13:55:58.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a00b27b0 0x7fa1a013ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:58.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa1a5cd7700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a00b3060 0x7fa1a0131820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa19c00b000 tx=0x7fa19c0014e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.865+0000 7fa18f7fe700 1 -- 172.21.15.117:0/1152154968 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa19c00a750 con 0x7fa1a00b3060 2020-11-20T13:55:58.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.866+0000 7fa18f7fe700 1 -- 172.21.15.117:0/1152154968 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c010040 con 0x7fa1a00b3060 2020-11-20T13:55:58.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.866+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a013b960 con 0x7fa1a00b3060 2020-11-20T13:55:58.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.866+0000 7fa18f7fe700 1 -- 172.21.15.117:0/1152154968 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa19c00aaa0 con 0x7fa1a00b3060 2020-11-20T13:55:58.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.866+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a0062ab0 con 0x7fa1a00b3060 2020-11-20T13:55:58.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.867+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a013baf0 con 0x7fa1a00b3060 2020-11-20T13:55:58.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.867+0000 7fa18f7fe700 1 -- 172.21.15.117:0/1152154968 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa19c01b020 con 0x7fa1a00b3060 2020-11-20T13:55:58.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.867+0000 7fa18f7fe700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa180051320 0x7fa1800537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:58.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.867+0000 7fa18f7fe700 1 -- 172.21.15.117:0/1152154968 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa19c068b10 con 0x7fa1a00b3060 2020-11-20T13:55:58.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.867+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa180051320 0x7fa1800537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:58.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.867+0000 7fa1a64d8700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa180051320 0x7fa1800537e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fa19000c000 tx=0x7fa19001a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:58.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:58.870+0000 7fa18f7fe700 1 -- 172.21.15.117:0/1152154968 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa1a013baf0 con 0x7fa1a00b3060 2020-11-20T13:55:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.020+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fa1a013baf0 con 0x7fa1a00b3060 2020-11-20T13:55:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.020+0000 7fa18f7fe700 1 -- 172.21.15.117:0/1152154968 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa1a013baf0 con 0x7fa1a00b3060 2020-11-20T13:55:59.022 INFO:teuthology.orchestra.run.smithi117.stdout:55834574854 2020-11-20T13:55:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa180051320 msgr2=0x7fa1800537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa180051320 0x7fa1800537e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fa19000c000 tx=0x7fa19001a000).stop 2020-11-20T13:55:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a00b3060 msgr2=0x7fa1a0131820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a00b3060 0x7fa1a0131820 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa19c00b000 tx=0x7fa19c0014e0).stop 2020-11-20T13:55:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 shutdown_connections 2020-11-20T13:55:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa180051320 0x7fa1800537e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1a00b27b0 0x7fa1a013ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa1a0131d10 0x7fa1a00b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 --2- 172.21.15.117:0/1152154968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a00b3060 0x7fa1a0131820 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 >> 172.21.15.117:0/1152154968 conn(0x7fa1a005b9f0 msgr2=0x7fa1a00b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 shutdown_connections 2020-11-20T13:55:59.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.022+0000 7fa1a873c700 1 -- 172.21.15.117:0/1152154968 wait complete. 2020-11-20T13:55:59.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2020-11-20T13:55:59.033 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2020-11-20T13:55:59.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.139+0000 7f6676911700 1 Processor -- start 2020-11-20T13:55:59.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.139+0000 7f6676911700 1 -- start start 2020-11-20T13:55:59.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f6676911700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f66700be620 0x7f6670146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f6676911700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66700b4a00 con 0x7f6670067b40 2020-11-20T13:55:59.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f6676911700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667014af20 con 0x7f66700b9d90 2020-11-20T13:55:59.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f6676911700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667014b1c0 con 0x7f66700be620 2020-11-20T13:55:59.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f6674eae700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f66700b9d90 0x7f66700b8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59632/0 (socket says 172.21.15.117:59632) 2020-11-20T13:55:59.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f6674eae700 1 -- 172.21.15.117:0/159005238 learned_addr learned my addr 172.21.15.117:0/159005238 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:59.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666ffff700 1 --2- 172.21.15.117:0/159005238 >> v2:172.21.15.85:3300/0 conn(0x7f66700be620 0x7f6670146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824956027 0 0) 0x7f667014af20 con 0x7f66700b9d90 2020-11-20T13:55:59.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f665c003230 con 0x7f66700b9d90 2020-11-20T13:55:59.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 860985128 0 0) 0x7f665c003230 con 0x7f66700b9d90 2020-11-20T13:55:59.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 >> v2:172.21.15.85:3300/0 conn(0x7f66700be620 msgr2=0x7f6670146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:59.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 --2- 172.21.15.117:0/159005238 >> v2:172.21.15.85:3300/0 conn(0x7f66700be620 0x7f6670146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 >> v1:172.21.15.85:6789/0 conn(0x7f6670067b40 legacy=0x7f66700b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667014b460 con 0x7f66700b9d90 2020-11-20T13:55:59.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f6668001a50 con 0x7f66700b9d90 2020-11-20T13:55:59.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.140+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 >> v1:172.21.15.117:6789/0 conn(0x7f66700b9d90 legacy=0x7f66700b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f665c003720 0x7f665c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f665c0044d0 0x7f665c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c004e40 0x7f665c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667014b1c0 con 0x7f665c004e40 2020-11-20T13:55:59.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c004340 con 0x7f665c0044d0 2020-11-20T13:55:59.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c00bc40 con 0x7f665c003720 2020-11-20T13:55:59.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666f7fe700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f665c0044d0 0x7f665c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666ffff700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f665c003720 0x7f665c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f6674eae700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c004e40 0x7f665c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666f7fe700 1 -- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f665c003720 msgr2=0x7f665c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:59.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666f7fe700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f665c003720 0x7f665c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666f7fe700 1 -- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c004e40 msgr2=0x7f665c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:59.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666f7fe700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c004e40 0x7f665c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f6674eae700 1 -- 172.21.15.117:0/159005238 reap_dead start 2020-11-20T13:55:59.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666f7fe700 1 -- 172.21.15.117:0/159005238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c003230 con 0x7f665c0044d0 2020-11-20T13:55:59.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666f7fe700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f665c0044d0 0x7f665c004950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f66700b8330 tx=0x7f6664007b60).ready entity=mon.1 client_cookie=38a52724e2c5fd19 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f666400c870 con 0x7f665c0044d0 2020-11-20T13:55:59.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666400c9d0 con 0x7f665c0044d0 2020-11-20T13:55:59.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f666effd700 1 -- 172.21.15.117:0/159005238 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f666400cd30 con 0x7f665c0044d0 2020-11-20T13:55:59.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f6676911700 1 -- 172.21.15.117:0/159005238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f665c0044d0 msgr2=0x7f665c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.141+0000 7f6676911700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f665c0044d0 0x7f665c004950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f66700b8330 tx=0x7f6664007b60).stop 2020-11-20T13:55:59.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.142+0000 7f6676911700 1 -- 172.21.15.117:0/159005238 shutdown_connections 2020-11-20T13:55:59.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.142+0000 7f6676911700 1 --2- 172.21.15.117:0/159005238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f665c0044d0 0x7f665c004950 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.142+0000 7f6676911700 1 -- 172.21.15.117:0/159005238 >> 172.21.15.117:0/159005238 conn(0x7f667005c4c0 msgr2=0x7f66700bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.142+0000 7f6676911700 1 -- 172.21.15.117:0/159005238 shutdown_connections 2020-11-20T13:55:59.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.142+0000 7f6676911700 1 -- 172.21.15.117:0/159005238 wait complete. 2020-11-20T13:55:59.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 Processor -- start 2020-11-20T13:55:59.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 -- start start 2020-11-20T13:55:59.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c0044d0 0x7f667014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f667008f7e0 0x7f66701484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66701562e0 0x7f66700b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66701599c0 con 0x7f665c0044d0 2020-11-20T13:55:59.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66701489d0 con 0x7f66701562e0 2020-11-20T13:55:59.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6676911700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6670148c60 con 0x7f667008f7e0 2020-11-20T13:55:59.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6674eae700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66701562e0 0x7f66700b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.143+0000 7f6674eae700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66701562e0 0x7f66700b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40484/0 (socket says 172.21.15.117:40484) 2020-11-20T13:55:59.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6674eae700 1 -- 172.21.15.117:0/1194121198 learned_addr learned my addr 172.21.15.117:0/1194121198 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:59.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f666f7fe700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f667008f7e0 0x7f66701484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f666ffff700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c0044d0 0x7f667014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6674eae700 1 -- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f667008f7e0 msgr2=0x7f66701484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6674eae700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f667008f7e0 0x7f66701484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6674eae700 1 -- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c0044d0 msgr2=0x7f667014a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6674eae700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c0044d0 0x7f667014a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6674eae700 1 -- 172.21.15.117:0/1194121198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670148f00 con 0x7f66701562e0 2020-11-20T13:55:59.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6674eae700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66701562e0 0x7f66700b4b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6668001cb0 tx=0x7f667014b5f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f666d7fa700 1 -- 172.21.15.117:0/1194121198 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f666800fb60 con 0x7f66701562e0 2020-11-20T13:55:59.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f666d7fa700 1 -- 172.21.15.117:0/1194121198 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666800fcc0 con 0x7f66701562e0 2020-11-20T13:55:59.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6670156790 con 0x7f66701562e0 2020-11-20T13:55:59.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f666d7fa700 1 -- 172.21.15.117:0/1194121198 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6668012350 con 0x7f66701562e0 2020-11-20T13:55:59.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.144+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66700b5080 con 0x7f66701562e0 2020-11-20T13:55:59.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.145+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6670057050 con 0x7f66701562e0 2020-11-20T13:55:59.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.145+0000 7f666d7fa700 1 -- 172.21.15.117:0/1194121198 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f66680124b0 con 0x7f66701562e0 2020-11-20T13:55:59.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.145+0000 7f666d7fa700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66500513c0 0x7f6650053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.145+0000 7f666d7fa700 1 -- 172.21.15.117:0/1194121198 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6668068910 con 0x7f66701562e0 2020-11-20T13:55:59.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.146+0000 7f666ffff700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66500513c0 0x7f6650053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.146+0000 7f666ffff700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66500513c0 0x7f6650053880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f6660008000 tx=0x7f6660008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.149+0000 7f666d7fa700 1 -- 172.21.15.117:0/1194121198 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6668040ad0 con 0x7f66701562e0 2020-11-20T13:55:59.301 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.300+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6670060540 con 0x7f66701562e0 2020-11-20T13:55:59.301 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.300+0000 7f666d7fa700 1 -- 172.21.15.117:0/1194121198 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6668019020 con 0x7f66701562e0 2020-11-20T13:55:59.302 INFO:teuthology.orchestra.run.smithi117.stdout:55834574854 2020-11-20T13:55:59.303 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.301+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66500513c0 msgr2=0x7f6650053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.303 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.301+0000 7f6676911700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66500513c0 0x7f6650053880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f6660008000 tx=0x7f6660008040).stop 2020-11-20T13:55:59.303 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.301+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66701562e0 msgr2=0x7f66700b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.303 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.301+0000 7f6676911700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66701562e0 0x7f66700b4b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6668001cb0 tx=0x7f667014b5f0).stop 2020-11-20T13:55:59.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 shutdown_connections 2020-11-20T13:55:59.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66500513c0 0x7f6650053880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66701562e0 0x7f66700b4b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f665c0044d0 0x7f667014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 --2- 172.21.15.117:0/1194121198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f667008f7e0 0x7f66701484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 >> 172.21.15.117:0/1194121198 conn(0x7f667005c4c0 msgr2=0x7f6670143b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 shutdown_connections 2020-11-20T13:55:59.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.302+0000 7f6676911700 1 -- 172.21.15.117:0/1194121198 wait complete. 2020-11-20T13:55:59.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2020-11-20T13:55:59.313 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-20T13:55:59.313 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-20T13:55:59.420 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.419+0000 7f8a75274700 1 Processor -- start 2020-11-20T13:55:59.421 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.419+0000 7f8a75274700 1 -- start start 2020-11-20T13:55:59.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a75274700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8a70098e30 0x7f8a7009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a75274700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a7009b700 0x7f8a701438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a75274700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70143dd0 0x7f8a700b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a75274700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700b5510 con 0x7f8a70098e30 2020-11-20T13:55:59.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a75274700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700b57b0 con 0x7f8a7009b700 2020-11-20T13:55:59.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a75274700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700b5a50 con 0x7f8a70143dd0 2020-11-20T13:55:59.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a7009b700 0x7f8a701438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a7009b700 0x7f8a701438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40488/0 (socket says 172.21.15.117:40488) 2020-11-20T13:55:59.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 -- 172.21.15.117:0/799141155 learned_addr learned my addr 172.21.15.117:0/799141155 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:59.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a6f7fe700 1 --2- 172.21.15.117:0/799141155 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70143dd0 0x7f8a700b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a6effd700 1 --2- 172.21.15.117:0/799141155 >> v2:172.21.15.85:3300/0 conn(0x7f8a70098e30 0x7f8a7009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 -- 172.21.15.117:0/799141155 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70143dd0 msgr2=0x7f8a700b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:59.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 --2- 172.21.15.117:0/799141155 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70143dd0 0x7f8a700b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 -- 172.21.15.117:0/799141155 >> v2:172.21.15.85:3300/0 conn(0x7f8a70098e30 msgr2=0x7f8a7009b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:59.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 --2- 172.21.15.117:0/799141155 >> v2:172.21.15.85:3300/0 conn(0x7f8a70098e30 0x7f8a7009b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 -- 172.21.15.117:0/799141155 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70099210 con 0x7f8a7009b700 2020-11-20T13:55:59.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.420+0000 7f8a67fff700 1 --2- 172.21.15.117:0/799141155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701438e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8a58002f30 tx=0x7f8a58006e80).ready entity=mon.1 client_cookie=91a43527a0acf4da server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a6e7fc700 1 -- 172.21.15.117:0/799141155 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a58009640 con 0x7f8a7009b700 2020-11-20T13:55:59.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a6e7fc700 1 -- 172.21.15.117:0/799141155 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a580097a0 con 0x7f8a7009b700 2020-11-20T13:55:59.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a6e7fc700 1 -- 172.21.15.117:0/799141155 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a58009b00 con 0x7f8a7009b700 2020-11-20T13:55:59.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 -- 172.21.15.117:0/799141155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 msgr2=0x7f8a701438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 --2- 172.21.15.117:0/799141155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701438e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f8a58002f30 tx=0x7f8a58006e80).stop 2020-11-20T13:55:59.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 -- 172.21.15.117:0/799141155 shutdown_connections 2020-11-20T13:55:59.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 --2- 172.21.15.117:0/799141155 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70143dd0 0x7f8a700b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 --2- 172.21.15.117:0/799141155 >> v2:172.21.15.85:3300/0 conn(0x7f8a70098e30 0x7f8a7009b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 --2- 172.21.15.117:0/799141155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701438e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 -- 172.21.15.117:0/799141155 >> 172.21.15.117:0/799141155 conn(0x7f8a7005b9f0 msgr2=0x7f8a7005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 -- 172.21.15.117:0/799141155 shutdown_connections 2020-11-20T13:55:59.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.421+0000 7f8a75274700 1 -- 172.21.15.117:0/799141155 wait complete. 2020-11-20T13:55:59.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.422+0000 7f8a75274700 1 Processor -- start 2020-11-20T13:55:59.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.422+0000 7f8a75274700 1 -- start start 2020-11-20T13:55:59.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70098e30 0x7f8a700b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8a70143dd0 0x7f8a70146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7013d510 con 0x7f8a70143dd0 2020-11-20T13:55:59.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70145ed0 con 0x7f8a7009b700 2020-11-20T13:55:59.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70146160 con 0x7f8a70098e30 2020-11-20T13:55:59.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40496/0 (socket says 172.21.15.117:40496) 2020-11-20T13:55:59.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 -- 172.21.15.117:0/2989029547 learned_addr learned my addr 172.21.15.117:0/2989029547 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:59.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a6effd700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70098e30 0x7f8a700b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a6f7fe700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8a70143dd0 0x7f8a70146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 -- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70098e30 msgr2=0x7f8a700b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70098e30 0x7f8a700b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 -- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8a70143dd0 msgr2=0x7f8a70146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8a70143dd0 0x7f8a70146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 -- 172.21.15.117:0/2989029547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70146400 con 0x7f8a7009b700 2020-11-20T13:55:59.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a6f7fe700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8a70143dd0 0x7f8a70146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:59.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a67fff700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701459e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8a70156320 tx=0x7f8a58015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a6cff9700 1 -- 172.21.15.117:0/2989029547 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a58011b40 con 0x7f8a7009b700 2020-11-20T13:55:59.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a6cff9700 1 -- 172.21.15.117:0/2989029547 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58011ca0 con 0x7f8a7009b700 2020-11-20T13:55:59.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a701466a0 con 0x7f8a7009b700 2020-11-20T13:55:59.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a6cff9700 1 -- 172.21.15.117:0/2989029547 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a5801d450 con 0x7f8a7009b700 2020-11-20T13:55:59.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.423+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a70159840 con 0x7f8a7009b700 2020-11-20T13:55:59.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.424+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70146830 con 0x7f8a7009b700 2020-11-20T13:55:59.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.424+0000 7f8a6cff9700 1 -- 172.21.15.117:0/2989029547 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8a5800f070 con 0x7f8a7009b700 2020-11-20T13:55:59.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.425+0000 7f8a6cff9700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8a54051230 0x7f8a540536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.425+0000 7f8a6cff9700 1 -- 172.21.15.117:0/2989029547 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f8a5806ab00 con 0x7f8a7009b700 2020-11-20T13:55:59.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.425+0000 7f8a6effd700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8a54051230 0x7f8a540536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.425+0000 7f8a6effd700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8a54051230 0x7f8a540536f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f8a6000b000 tx=0x7f8a60006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.428+0000 7f8a6cff9700 1 -- 172.21.15.117:0/2989029547 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8a58042d20 con 0x7f8a7009b700 2020-11-20T13:55:59.574 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.573+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 --> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a70057050 con 0x7f8a54051230 2020-11-20T13:55:59.578 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.576+0000 7f8a6cff9700 1 -- 172.21.15.117:0/2989029547 <== mgr.4111 v2:172.21.15.117:6800/22275 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82333 (secure 0 0 0) 0x7f8a70057050 con 0x7f8a54051230 2020-11-20T13:55:59.578 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:59.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8a54051230 msgr2=0x7f8a540536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8a54051230 0x7f8a540536f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f8a6000b000 tx=0x7f8a60006dc0).stop 2020-11-20T13:55:59.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 msgr2=0x7f8a701459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701459e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f8a70156320 tx=0x7f8a58015000).stop 2020-11-20T13:55:59.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 shutdown_connections 2020-11-20T13:55:59.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8a54051230 0x7f8a540536f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8a70098e30 0x7f8a700b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8a70143dd0 0x7f8a70146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 --2- 172.21.15.117:0/2989029547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a7009b700 0x7f8a701459e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 >> 172.21.15.117:0/2989029547 conn(0x7f8a7005b9f0 msgr2=0x7f8a7009bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 shutdown_connections 2020-11-20T13:55:59.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.578+0000 7f8a75274700 1 -- 172.21.15.117:0/2989029547 wait complete. 2020-11-20T13:55:59.582 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2020-11-20T13:55:59.589 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2020-11-20T13:55:58.138281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390816,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583904,"statfs":{"total":773094113280,"available":764501917696,"internally_reserved":8589934592,"allocated":2195456,"data_stored":873072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.815585"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.697099+0000","last_change":"2020-11-20T13:55:50.695895+0000","last_active":"2020-11-20T13:55:50.697099+0000","last_peered":"2020-11-20T13:55:50.697099+0000","last_clean":"2020-11-20T13:55:50.697099+0000","last_became_active":"2020-11-20T13:55:50.694279+0000","last_became_peered":"2020-11-20T13:55:50.694279+0000","last_unstale":"2020-11-20T13:55:50.697099+0000","last_undegraded":"2020-11-20T13:55:50.697099+0000","last_fullsized":"2020-11-20T13:55:50.697099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.685613+0000","last_change":"2020-11-20T13:55:44.215412+0000","last_active":"2020-11-20T13:55:50.685613+0000","last_peered":"2020-11-20T13:55:50.685613+0000","last_clean":"2020-11-20T13:55:50.685613+0000","last_became_active":"2020-11-20T13:55:44.215026+0000","last_became_peered":"2020-11-20T13:55:44.215026+0000","last_unstale":"2020-11-20T13:55:50.685613+0000","last_undegraded":"2020-11-20T13:55:50.685613+0000","last_fullsized":"2020-11-20T13:55:50.685613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.695669+0000","last_change":"2020-11-20T13:55:47.315560+0000","last_active":"2020-11-20T13:55:50.695669+0000","last_peered":"2020-11-20T13:55:50.695669+0000","last_clean":"2020-11-20T13:55:50.695669+0000","last_became_active":"2020-11-20T13:55:47.315133+0000","last_became_peered":"2020-11-20T13:55:47.315133+0000","last_unstale":"2020-11-20T13:55:50.695669+0000","last_undegraded":"2020-11-20T13:55:50.695669+0000","last_fullsized":"2020-11-20T13:55:50.695669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.685215+0000","last_change":"2020-11-20T13:55:50.333373+0000","last_active":"2020-11-20T13:55:50.685215+0000","last_peered":"2020-11-20T13:55:50.685215+0000","last_clean":"2020-11-20T13:55:50.685215+0000","last_became_active":"2020-11-20T13:55:50.332783+0000","last_became_peered":"2020-11-20T13:55:50.332783+0000","last_unstale":"2020-11-20T13:55:50.685215+0000","last_undegraded":"2020-11-20T13:55:50.685215+0000","last_fullsized":"2020-11-20T13:55:50.685215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686220+0000","last_change":"2020-11-20T13:55:44.198568+0000","last_active":"2020-11-20T13:55:50.686220+0000","last_peered":"2020-11-20T13:55:50.686220+0000","last_clean":"2020-11-20T13:55:50.686220+0000","last_became_active":"2020-11-20T13:55:44.198268+0000","last_became_peered":"2020-11-20T13:55:44.198268+0000","last_unstale":"2020-11-20T13:55:50.686220+0000","last_undegraded":"2020-11-20T13:55:50.686220+0000","last_fullsized":"2020-11-20T13:55:50.686220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.697702+0000","last_change":"2020-11-20T13:55:47.317955+0000","last_active":"2020-11-20T13:55:50.697702+0000","last_peered":"2020-11-20T13:55:50.697702+0000","last_clean":"2020-11-20T13:55:50.697702+0000","last_became_active":"2020-11-20T13:55:47.317551+0000","last_became_peered":"2020-11-20T13:55:47.317551+0000","last_unstale":"2020-11-20T13:55:50.697702+0000","last_undegraded":"2020-11-20T13:55:50.697702+0000","last_fullsized":"2020-11-20T13:55:50.697702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.697066+0000","last_change":"2020-11-20T13:55:50.695493+0000","last_active":"2020-11-20T13:55:50.697066+0000","last_peered":"2020-11-20T13:55:50.697066+0000","last_clean":"2020-11-20T13:55:50.697066+0000","last_became_active":"2020-11-20T13:55:50.694932+0000","last_became_peered":"2020-11-20T13:55:50.694932+0000","last_unstale":"2020-11-20T13:55:50.697066+0000","last_undegraded":"2020-11-20T13:55:50.697066+0000","last_fullsized":"2020-11-20T13:55:50.697066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.679627+0000","last_change":"2020-11-20T13:55:44.259549+0000","last_active":"2020-11-20T13:55:50.679627+0000","last_peered":"2020-11-20T13:55:50.679627+0000","last_clean":"2020-11-20T13:55:50.679627+0000","last_became_active":"2020-11-20T13:55:44.259087+0000","last_became_peered":"2020-11-20T13:55:44.259087+0000","last_unstale":"2020-11-20T13:55:50.679627+0000","last_undegraded":"2020-11-20T13:55:50.679627+0000","last_fullsized":"2020-11-20T13:55:50.679627+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":"21","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.698547+0000","last_change":"2020-11-20T13:55:47.818825+0000","last_active":"2020-11-20T13:55:50.698547+0000","last_peered":"2020-11-20T13:55:50.698547+0000","last_clean":"2020-11-20T13:55:50.698547+0000","last_became_active":"2020-11-20T13:55:47.818467+0000","last_became_peered":"2020-11-20T13:55:47.818467+0000","last_unstale":"2020-11-20T13:55:50.698547+0000","last_undegraded":"2020-11-20T13:55:50.698547+0000","last_fullsized":"2020-11-20T13:55:50.698547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.699865+0000","last_change":"2020-11-20T13:55:50.694411+0000","last_active":"2020-11-20T13:55:50.699865+0000","last_peered":"2020-11-20T13:55:50.699865+0000","last_clean":"2020-11-20T13:55:50.699865+0000","last_became_active":"2020-11-20T13:55:50.693536+0000","last_became_peered":"2020-11-20T13:55:50.693536+0000","last_unstale":"2020-11-20T13:55:50.699865+0000","last_undegraded":"2020-11-20T13:55:50.699865+0000","last_fullsized":"2020-11-20T13:55:50.699865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686242+0000","last_change":"2020-11-20T13:55:44.199082+0000","last_active":"2020-11-20T13:55:50.686242+0000","last_peered":"2020-11-20T13:55:50.686242+0000","last_clean":"2020-11-20T13:55:50.686242+0000","last_became_active":"2020-11-20T13:55:44.198784+0000","last_became_peered":"2020-11-20T13:55:44.198784+0000","last_unstale":"2020-11-20T13:55:50.686242+0000","last_undegraded":"2020-11-20T13:55:50.686242+0000","last_fullsized":"2020-11-20T13:55:50.686242+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686817+0000","last_change":"2020-11-20T13:55:40.287162+0000","last_active":"2020-11-20T13:55:50.686817+0000","last_peered":"2020-11-20T13:55:50.686817+0000","last_clean":"2020-11-20T13:55:50.686817+0000","last_became_active":"2020-11-20T13:55:40.286439+0000","last_became_peered":"2020-11-20T13:55:40.286439+0000","last_unstale":"2020-11-20T13:55:50.686817+0000","last_undegraded":"2020-11-20T13:55:50.686817+0000","last_fullsized":"2020-11-20T13:55:50.686817+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:34.290777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:34.290777+0000","last_clean_scrub_stamp":"2020-11-20T13:55:34.290777+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.684381+0000","last_change":"2020-11-20T13:55:47.316867+0000","last_active":"2020-11-20T13:55:50.684381+0000","last_peered":"2020-11-20T13:55:50.684381+0000","last_clean":"2020-11-20T13:55:50.684381+0000","last_became_active":"2020-11-20T13:55:47.316482+0000","last_became_peered":"2020-11-20T13:55:47.316482+0000","last_unstale":"2020-11-20T13:55:50.684381+0000","last_undegraded":"2020-11-20T13:55:50.684381+0000","last_fullsized":"2020-11-20T13:55:50.684381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.695999+0000","last_change":"2020-11-20T13:55:50.692096+0000","last_active":"2020-11-20T13:55:50.695999+0000","last_peered":"2020-11-20T13:55:50.695999+0000","last_clean":"2020-11-20T13:55:50.695999+0000","last_became_active":"2020-11-20T13:55:50.691570+0000","last_became_peered":"2020-11-20T13:55:50.691570+0000","last_unstale":"2020-11-20T13:55:50.695999+0000","last_undegraded":"2020-11-20T13:55:50.695999+0000","last_fullsized":"2020-11-20T13:55:50.695999+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686984+0000","last_change":"2020-11-20T13:55:44.192220+0000","last_active":"2020-11-20T13:55:50.686984+0000","last_peered":"2020-11-20T13:55:50.686984+0000","last_clean":"2020-11-20T13:55:50.686984+0000","last_became_active":"2020-11-20T13:55:44.191963+0000","last_became_peered":"2020-11-20T13:55:44.191963+0000","last_unstale":"2020-11-20T13:55:50.686984+0000","last_undegraded":"2020-11-20T13:55:50.686984+0000","last_fullsized":"2020-11-20T13:55:50.686984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"13","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.685222+0000","last_change":"2020-11-20T13:55:47.316982+0000","last_active":"2020-11-20T13:55:50.685222+0000","last_peered":"2020-11-20T13:55:50.685222+0000","last_clean":"2020-11-20T13:55:50.685222+0000","last_became_active":"2020-11-20T13:55:47.316530+0000","last_became_peered":"2020-11-20T13:55:47.316530+0000","last_unstale":"2020-11-20T13:55:50.685222+0000","last_undegraded":"2020-11-20T13:55:50.685222+0000","last_fullsized":"2020-11-20T13:55:50.685222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.696608+0000","last_change":"2020-11-20T13:55:50.694956+0000","last_active":"2020-11-20T13:55:50.696608+0000","last_peered":"2020-11-20T13:55:50.696608+0000","last_clean":"2020-11-20T13:55:50.696608+0000","last_became_active":"2020-11-20T13:55:50.693966+0000","last_became_peered":"2020-11-20T13:55:50.693966+0000","last_unstale":"2020-11-20T13:55:50.696608+0000","last_undegraded":"2020-11-20T13:55:50.696608+0000","last_fullsized":"2020-11-20T13:55:50.696608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.687223+0000","last_change":"2020-11-20T13:55:44.214932+0000","last_active":"2020-11-20T13:55:50.687223+0000","last_peered":"2020-11-20T13:55:50.687223+0000","last_clean":"2020-11-20T13:55:50.687223+0000","last_became_active":"2020-11-20T13:55:44.214492+0000","last_became_peered":"2020-11-20T13:55:44.214492+0000","last_unstale":"2020-11-20T13:55:50.687223+0000","last_undegraded":"2020-11-20T13:55:50.687223+0000","last_fullsized":"2020-11-20T13:55:50.687223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":"21","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.703742+0000","last_change":"2020-11-20T13:55:47.818676+0000","last_active":"2020-11-20T13:55:50.703742+0000","last_peered":"2020-11-20T13:55:50.703742+0000","last_clean":"2020-11-20T13:55:50.703742+0000","last_became_active":"2020-11-20T13:55:47.818355+0000","last_became_peered":"2020-11-20T13:55:47.818355+0000","last_unstale":"2020-11-20T13:55:50.703742+0000","last_undegraded":"2020-11-20T13:55:50.703742+0000","last_fullsized":"2020-11-20T13:55:50.703742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.693259+0000","last_change":"2020-11-20T13:55:50.691886+0000","last_active":"2020-11-20T13:55:50.693259+0000","last_peered":"2020-11-20T13:55:50.693259+0000","last_clean":"2020-11-20T13:55:50.693259+0000","last_became_active":"2020-11-20T13:55:50.691328+0000","last_became_peered":"2020-11-20T13:55:50.691328+0000","last_unstale":"2020-11-20T13:55:50.693259+0000","last_undegraded":"2020-11-20T13:55:50.693259+0000","last_fullsized":"2020-11-20T13:55:50.693259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"24'7","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.694810+0000","last_change":"2020-11-20T13:55:47.821416+0000","last_active":"2020-11-20T13:55:50.694810+0000","last_peered":"2020-11-20T13:55:50.694810+0000","last_clean":"2020-11-20T13:55:50.694810+0000","last_became_active":"2020-11-20T13:55:47.821069+0000","last_became_peered":"2020-11-20T13:55:47.821069+0000","last_unstale":"2020-11-20T13:55:50.694810+0000","last_undegraded":"2020-11-20T13:55:50.694810+0000","last_fullsized":"2020-11-20T13:55:50.694810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.687273+0000","last_change":"2020-11-20T13:55:44.202395+0000","last_active":"2020-11-20T13:55:50.687273+0000","last_peered":"2020-11-20T13:55:50.687273+0000","last_clean":"2020-11-20T13:55:50.687273+0000","last_became_active":"2020-11-20T13:55:44.201839+0000","last_became_peered":"2020-11-20T13:55:44.201839+0000","last_unstale":"2020-11-20T13:55:50.687273+0000","last_undegraded":"2020-11-20T13:55:50.687273+0000","last_fullsized":"2020-11-20T13:55:50.687273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.699977+0000","last_change":"2020-11-20T13:55:50.694156+0000","last_active":"2020-11-20T13:55:50.699977+0000","last_peered":"2020-11-20T13:55:50.699977+0000","last_clean":"2020-11-20T13:55:50.699977+0000","last_became_active":"2020-11-20T13:55:50.693499+0000","last_became_peered":"2020-11-20T13:55:50.693499+0000","last_unstale":"2020-11-20T13:55:50.699977+0000","last_undegraded":"2020-11-20T13:55:50.699977+0000","last_fullsized":"2020-11-20T13:55:50.699977+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.692825+0000","last_change":"2020-11-20T13:55:44.201759+0000","last_active":"2020-11-20T13:55:50.692825+0000","last_peered":"2020-11-20T13:55:50.692825+0000","last_clean":"2020-11-20T13:55:50.692825+0000","last_became_active":"2020-11-20T13:55:44.201419+0000","last_became_peered":"2020-11-20T13:55:44.201419+0000","last_unstale":"2020-11-20T13:55:50.692825+0000","last_undegraded":"2020-11-20T13:55:50.692825+0000","last_fullsized":"2020-11-20T13:55:50.692825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T13:55:51.898189+0000","last_change":"2020-11-20T13:55:47.323565+0000","last_active":"2020-11-20T13:55:51.898189+0000","last_peered":"2020-11-20T13:55:51.898189+0000","last_clean":"2020-11-20T13:55:51.898189+0000","last_became_active":"2020-11-20T13:55:47.323216+0000","last_became_peered":"2020-11-20T13:55:47.323216+0000","last_unstale":"2020-11-20T13:55:51.898189+0000","last_undegraded":"2020-11-20T13:55:51.898189+0000","last_fullsized":"2020-11-20T13:55:51.898189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":109133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":108558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":108592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":109293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":108592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":108718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]}]}]}} 2020-11-20T13:55:59.591 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-11-20T13:55:59.697 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.695+0000 7f3b9427c700 1 Processor -- start 2020-11-20T13:55:59.697 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.695+0000 7f3b9427c700 1 -- start start 2020-11-20T13:55:59.699 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.696+0000 7f3b9427c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f3b8c147190 0x7f3b8c148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.700 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.696+0000 7f3b9427c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b8c09aea0 0x7f3b8c061f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.700 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.696+0000 7f3b9427c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8c147880 con 0x7f3b8c0b8b50 2020-11-20T13:55:59.700 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.696+0000 7f3b9427c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8c147b20 con 0x7f3b8c147190 2020-11-20T13:55:59.700 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.696+0000 7f3b9427c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8c147dc0 con 0x7f3b8c09aea0 2020-11-20T13:55:59.700 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.696+0000 7f3b91817700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b8c09aea0 0x7f3b8c061f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91817700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b8c09aea0 0x7f3b8c061f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40506/0 (socket says 172.21.15.117:40506) 2020-11-20T13:55:59.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b92819700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3b8c0b8b50 0x7f3b8c14acb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59664/0 (socket says 172.21.15.117:59664) 2020-11-20T13:55:59.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91817700 1 -- 172.21.15.117:0/2548115698 learned_addr learned my addr 172.21.15.117:0/2548115698 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:59.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b92018700 1 --2- 172.21.15.117:0/2548115698 >> v2:172.21.15.85:3300/0 conn(0x7f3b8c147190 0x7f3b8c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91817700 1 -- 172.21.15.117:0/2548115698 >> v2:172.21.15.85:3300/0 conn(0x7f3b8c147190 msgr2=0x7f3b8c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:55:59.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91817700 1 --2- 172.21.15.117:0/2548115698 >> v2:172.21.15.85:3300/0 conn(0x7f3b8c147190 0x7f3b8c148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91817700 1 -- 172.21.15.117:0/2548115698 >> v1:172.21.15.117:6789/0 conn(0x7f3b8c0b8b50 legacy=0x7f3b8c14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91016700 1 -- 172.21.15.117:0/2548115698 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965085454 0 0) 0x7f3b8c147880 con 0x7f3b8c0b8b50 2020-11-20T13:55:59.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91817700 1 -- 172.21.15.117:0/2548115698 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c147570 con 0x7f3b8c09aea0 2020-11-20T13:55:59.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b92018700 1 --2- 172.21.15.117:0/2548115698 >> v2:172.21.15.85:3300/0 conn(0x7f3b8c147190 0x7f3b8c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:59.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91817700 1 --2- 172.21.15.117:0/2548115698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c061f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f3b88002f30 tx=0x7f3b88006e80).ready entity=mon.1 client_cookie=d11b6fa37c1256be server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91016700 1 -- 172.21.15.117:0/2548115698 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b880097f0 con 0x7f3b8c09aea0 2020-11-20T13:55:59.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91016700 1 -- 172.21.15.117:0/2548115698 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b88009950 con 0x7f3b8c09aea0 2020-11-20T13:55:59.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b91016700 1 -- 172.21.15.117:0/2548115698 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b88009cb0 con 0x7f3b8c09aea0 2020-11-20T13:55:59.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b9427c700 1 -- 172.21.15.117:0/2548115698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 msgr2=0x7f3b8c061f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.697+0000 7f3b9427c700 1 --2- 172.21.15.117:0/2548115698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c061f40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f3b88002f30 tx=0x7f3b88006e80).stop 2020-11-20T13:55:59.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.698+0000 7f3b9427c700 1 -- 172.21.15.117:0/2548115698 shutdown_connections 2020-11-20T13:55:59.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.698+0000 7f3b9427c700 1 --2- 172.21.15.117:0/2548115698 >> v2:172.21.15.85:3300/0 conn(0x7f3b8c147190 0x7f3b8c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.698+0000 7f3b9427c700 1 --2- 172.21.15.117:0/2548115698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c061f40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.698+0000 7f3b9427c700 1 -- 172.21.15.117:0/2548115698 >> 172.21.15.117:0/2548115698 conn(0x7f3b8c05c4c0 msgr2=0x7f3b8c0b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.698+0000 7f3b9427c700 1 -- 172.21.15.117:0/2548115698 shutdown_connections 2020-11-20T13:55:59.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.698+0000 7f3b9427c700 1 -- 172.21.15.117:0/2548115698 wait complete. 2020-11-20T13:55:59.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 Processor -- start 2020-11-20T13:55:59.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 -- start start 2020-11-20T13:55:59.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c0649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.709 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b8c0b8b50 0x7f3b8c0ba610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.709 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b8c147190 0x7f3b8c142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.709 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c09b870 con 0x7f3b8c147190 2020-11-20T13:55:59.710 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c09c9e0 con 0x7f3b8c09aea0 2020-11-20T13:55:59.710 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b9427c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c09b5d0 con 0x7f3b8c0b8b50 2020-11-20T13:55:59.710 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b92018700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c0649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.710 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b92018700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c0649a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40508/0 (socket says 172.21.15.117:40508) 2020-11-20T13:55:59.710 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b92018700 1 -- 172.21.15.117:0/3673691738 learned_addr learned my addr 172.21.15.117:0/3673691738 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:59.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b91817700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b8c0b8b50 0x7f3b8c0ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.699+0000 7f3b92819700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b8c147190 0x7f3b8c142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b92018700 1 -- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b8c0b8b50 msgr2=0x7f3b8c0ba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b92018700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b8c0b8b50 0x7f3b8c0ba610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b92018700 1 -- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b8c147190 msgr2=0x7f3b8c142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b92018700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b8c147190 0x7f3b8c142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b92018700 1 -- 172.21.15.117:0/3673691738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c0bad70 con 0x7f3b8c09aea0 2020-11-20T13:55:59.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b92819700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b8c147190 0x7f3b8c142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:55:59.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b92018700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c0649a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f3b8c14a2c0 tx=0x7f3b7c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b7b7fe700 1 -- 172.21.15.117:0/3673691738 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7c009980 con 0x7f3b8c09aea0 2020-11-20T13:55:59.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b7b7fe700 1 -- 172.21.15.117:0/3673691738 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c009ae0 con 0x7f3b8c09aea0 2020-11-20T13:55:59.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b7b7fe700 1 -- 172.21.15.117:0/3673691738 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7c009e40 con 0x7f3b8c09aea0 2020-11-20T13:55:59.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c09c3e0 con 0x7f3b8c09aea0 2020-11-20T13:55:59.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.700+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c09dfe0 con 0x7f3b8c09aea0 2020-11-20T13:55:59.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.701+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c057050 con 0x7f3b8c09aea0 2020-11-20T13:55:59.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.701+0000 7f3b7b7fe700 1 -- 172.21.15.117:0/3673691738 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3b7c013480 con 0x7f3b8c09aea0 2020-11-20T13:55:59.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.702+0000 7f3b7b7fe700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3b6c051290 0x7f3b6c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.702+0000 7f3b7b7fe700 1 -- 172.21.15.117:0/3673691738 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3b7c069970 con 0x7f3b8c09aea0 2020-11-20T13:55:59.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.702+0000 7f3b91817700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3b6c051290 0x7f3b6c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.702+0000 7f3b91817700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3b6c051290 0x7f3b6c053750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3b8801c000 tx=0x7f3b88015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.705+0000 7f3b7b7fe700 1 -- 172.21.15.117:0/3673691738 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3b7c041b30 con 0x7f3b8c09aea0 2020-11-20T13:55:59.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.849+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 --> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b8c09c860 con 0x7f3b6c051290 2020-11-20T13:55:59.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.852+0000 7f3b7b7fe700 1 -- 172.21.15.117:0/3673691738 <== mgr.4111 v2:172.21.15.117:6800/22275 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82333 (secure 0 0 0) 0x7f3b88006910 con 0x7f3b6c051290 2020-11-20T13:55:59.854 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:55:59.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3b6c051290 msgr2=0x7f3b6c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3b6c051290 0x7f3b6c053750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3b8801c000 tx=0x7f3b88015000).stop 2020-11-20T13:55:59.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 msgr2=0x7f3b8c0649a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c0649a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f3b8c14a2c0 tx=0x7f3b7c008040).stop 2020-11-20T13:55:59.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 shutdown_connections 2020-11-20T13:55:59.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3b6c051290 0x7f3b6c053750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b8c147190 0x7f3b8c142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c09aea0 0x7f3b8c0649a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 --2- 172.21.15.117:0/3673691738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b8c0b8b50 0x7f3b8c0ba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 >> 172.21.15.117:0/3673691738 conn(0x7f3b8c05c4c0 msgr2=0x7f3b8c0b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.853+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 shutdown_connections 2020-11-20T13:55:59.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.854+0000 7f3b9427c700 1 -- 172.21.15.117:0/3673691738 wait complete. 2020-11-20T13:55:59.858 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2020-11-20T13:55:59.865 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2020-11-20T13:55:58.138281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390816,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746583904,"statfs":{"total":773094113280,"available":764501917696,"internally_reserved":8589934592,"allocated":2195456,"data_stored":873072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.815585"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.697099+0000","last_change":"2020-11-20T13:55:50.695895+0000","last_active":"2020-11-20T13:55:50.697099+0000","last_peered":"2020-11-20T13:55:50.697099+0000","last_clean":"2020-11-20T13:55:50.697099+0000","last_became_active":"2020-11-20T13:55:50.694279+0000","last_became_peered":"2020-11-20T13:55:50.694279+0000","last_unstale":"2020-11-20T13:55:50.697099+0000","last_undegraded":"2020-11-20T13:55:50.697099+0000","last_fullsized":"2020-11-20T13:55:50.697099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.685613+0000","last_change":"2020-11-20T13:55:44.215412+0000","last_active":"2020-11-20T13:55:50.685613+0000","last_peered":"2020-11-20T13:55:50.685613+0000","last_clean":"2020-11-20T13:55:50.685613+0000","last_became_active":"2020-11-20T13:55:44.215026+0000","last_became_peered":"2020-11-20T13:55:44.215026+0000","last_unstale":"2020-11-20T13:55:50.685613+0000","last_undegraded":"2020-11-20T13:55:50.685613+0000","last_fullsized":"2020-11-20T13:55:50.685613+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'6","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.695669+0000","last_change":"2020-11-20T13:55:47.315560+0000","last_active":"2020-11-20T13:55:50.695669+0000","last_peered":"2020-11-20T13:55:50.695669+0000","last_clean":"2020-11-20T13:55:50.695669+0000","last_became_active":"2020-11-20T13:55:47.315133+0000","last_became_peered":"2020-11-20T13:55:47.315133+0000","last_unstale":"2020-11-20T13:55:50.695669+0000","last_undegraded":"2020-11-20T13:55:50.695669+0000","last_fullsized":"2020-11-20T13:55:50.695669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.685215+0000","last_change":"2020-11-20T13:55:50.333373+0000","last_active":"2020-11-20T13:55:50.685215+0000","last_peered":"2020-11-20T13:55:50.685215+0000","last_clean":"2020-11-20T13:55:50.685215+0000","last_became_active":"2020-11-20T13:55:50.332783+0000","last_became_peered":"2020-11-20T13:55:50.332783+0000","last_unstale":"2020-11-20T13:55:50.685215+0000","last_undegraded":"2020-11-20T13:55:50.685215+0000","last_fullsized":"2020-11-20T13:55:50.685215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686220+0000","last_change":"2020-11-20T13:55:44.198568+0000","last_active":"2020-11-20T13:55:50.686220+0000","last_peered":"2020-11-20T13:55:50.686220+0000","last_clean":"2020-11-20T13:55:50.686220+0000","last_became_active":"2020-11-20T13:55:44.198268+0000","last_became_peered":"2020-11-20T13:55:44.198268+0000","last_unstale":"2020-11-20T13:55:50.686220+0000","last_undegraded":"2020-11-20T13:55:50.686220+0000","last_fullsized":"2020-11-20T13:55:50.686220+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'5","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.697702+0000","last_change":"2020-11-20T13:55:47.317955+0000","last_active":"2020-11-20T13:55:50.697702+0000","last_peered":"2020-11-20T13:55:50.697702+0000","last_clean":"2020-11-20T13:55:50.697702+0000","last_became_active":"2020-11-20T13:55:47.317551+0000","last_became_peered":"2020-11-20T13:55:47.317551+0000","last_unstale":"2020-11-20T13:55:50.697702+0000","last_undegraded":"2020-11-20T13:55:50.697702+0000","last_fullsized":"2020-11-20T13:55:50.697702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.697066+0000","last_change":"2020-11-20T13:55:50.695493+0000","last_active":"2020-11-20T13:55:50.697066+0000","last_peered":"2020-11-20T13:55:50.697066+0000","last_clean":"2020-11-20T13:55:50.697066+0000","last_became_active":"2020-11-20T13:55:50.694932+0000","last_became_peered":"2020-11-20T13:55:50.694932+0000","last_unstale":"2020-11-20T13:55:50.697066+0000","last_undegraded":"2020-11-20T13:55:50.697066+0000","last_fullsized":"2020-11-20T13:55:50.697066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.679627+0000","last_change":"2020-11-20T13:55:44.259549+0000","last_active":"2020-11-20T13:55:50.679627+0000","last_peered":"2020-11-20T13:55:50.679627+0000","last_clean":"2020-11-20T13:55:50.679627+0000","last_became_active":"2020-11-20T13:55:44.259087+0000","last_became_peered":"2020-11-20T13:55:44.259087+0000","last_unstale":"2020-11-20T13:55:50.679627+0000","last_undegraded":"2020-11-20T13:55:50.679627+0000","last_fullsized":"2020-11-20T13:55:50.679627+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"24'8","reported_seq":"21","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.698547+0000","last_change":"2020-11-20T13:55:47.818825+0000","last_active":"2020-11-20T13:55:50.698547+0000","last_peered":"2020-11-20T13:55:50.698547+0000","last_clean":"2020-11-20T13:55:50.698547+0000","last_became_active":"2020-11-20T13:55:47.818467+0000","last_became_peered":"2020-11-20T13:55:47.818467+0000","last_unstale":"2020-11-20T13:55:50.698547+0000","last_undegraded":"2020-11-20T13:55:50.698547+0000","last_fullsized":"2020-11-20T13:55:50.698547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.699865+0000","last_change":"2020-11-20T13:55:50.694411+0000","last_active":"2020-11-20T13:55:50.699865+0000","last_peered":"2020-11-20T13:55:50.699865+0000","last_clean":"2020-11-20T13:55:50.699865+0000","last_became_active":"2020-11-20T13:55:50.693536+0000","last_became_peered":"2020-11-20T13:55:50.693536+0000","last_unstale":"2020-11-20T13:55:50.699865+0000","last_undegraded":"2020-11-20T13:55:50.699865+0000","last_fullsized":"2020-11-20T13:55:50.699865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686242+0000","last_change":"2020-11-20T13:55:44.199082+0000","last_active":"2020-11-20T13:55:50.686242+0000","last_peered":"2020-11-20T13:55:50.686242+0000","last_clean":"2020-11-20T13:55:50.686242+0000","last_became_active":"2020-11-20T13:55:44.198784+0000","last_became_peered":"2020-11-20T13:55:44.198784+0000","last_unstale":"2020-11-20T13:55:50.686242+0000","last_undegraded":"2020-11-20T13:55:50.686242+0000","last_fullsized":"2020-11-20T13:55:50.686242+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686817+0000","last_change":"2020-11-20T13:55:40.287162+0000","last_active":"2020-11-20T13:55:50.686817+0000","last_peered":"2020-11-20T13:55:50.686817+0000","last_clean":"2020-11-20T13:55:50.686817+0000","last_became_active":"2020-11-20T13:55:40.286439+0000","last_became_peered":"2020-11-20T13:55:40.286439+0000","last_unstale":"2020-11-20T13:55:50.686817+0000","last_undegraded":"2020-11-20T13:55:50.686817+0000","last_fullsized":"2020-11-20T13:55:50.686817+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:34.290777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:34.290777+0000","last_clean_scrub_stamp":"2020-11-20T13:55:34.290777+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'1","reported_seq":"14","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.684381+0000","last_change":"2020-11-20T13:55:47.316867+0000","last_active":"2020-11-20T13:55:50.684381+0000","last_peered":"2020-11-20T13:55:50.684381+0000","last_clean":"2020-11-20T13:55:50.684381+0000","last_became_active":"2020-11-20T13:55:47.316482+0000","last_became_peered":"2020-11-20T13:55:47.316482+0000","last_unstale":"2020-11-20T13:55:50.684381+0000","last_undegraded":"2020-11-20T13:55:50.684381+0000","last_fullsized":"2020-11-20T13:55:50.684381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.695999+0000","last_change":"2020-11-20T13:55:50.692096+0000","last_active":"2020-11-20T13:55:50.695999+0000","last_peered":"2020-11-20T13:55:50.695999+0000","last_clean":"2020-11-20T13:55:50.695999+0000","last_became_active":"2020-11-20T13:55:50.691570+0000","last_became_peered":"2020-11-20T13:55:50.691570+0000","last_unstale":"2020-11-20T13:55:50.695999+0000","last_undegraded":"2020-11-20T13:55:50.695999+0000","last_fullsized":"2020-11-20T13:55:50.695999+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.686984+0000","last_change":"2020-11-20T13:55:44.192220+0000","last_active":"2020-11-20T13:55:50.686984+0000","last_peered":"2020-11-20T13:55:50.686984+0000","last_clean":"2020-11-20T13:55:50.686984+0000","last_became_active":"2020-11-20T13:55:44.191963+0000","last_became_peered":"2020-11-20T13:55:44.191963+0000","last_unstale":"2020-11-20T13:55:50.686984+0000","last_undegraded":"2020-11-20T13:55:50.686984+0000","last_fullsized":"2020-11-20T13:55:50.686984+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"13","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.685222+0000","last_change":"2020-11-20T13:55:47.316982+0000","last_active":"2020-11-20T13:55:50.685222+0000","last_peered":"2020-11-20T13:55:50.685222+0000","last_clean":"2020-11-20T13:55:50.685222+0000","last_became_active":"2020-11-20T13:55:47.316530+0000","last_became_peered":"2020-11-20T13:55:47.316530+0000","last_unstale":"2020-11-20T13:55:50.685222+0000","last_undegraded":"2020-11-20T13:55:50.685222+0000","last_fullsized":"2020-11-20T13:55:50.685222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.696608+0000","last_change":"2020-11-20T13:55:50.694956+0000","last_active":"2020-11-20T13:55:50.696608+0000","last_peered":"2020-11-20T13:55:50.696608+0000","last_clean":"2020-11-20T13:55:50.696608+0000","last_became_active":"2020-11-20T13:55:50.693966+0000","last_became_peered":"2020-11-20T13:55:50.693966+0000","last_unstale":"2020-11-20T13:55:50.696608+0000","last_undegraded":"2020-11-20T13:55:50.696608+0000","last_fullsized":"2020-11-20T13:55:50.696608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.687223+0000","last_change":"2020-11-20T13:55:44.214932+0000","last_active":"2020-11-20T13:55:50.687223+0000","last_peered":"2020-11-20T13:55:50.687223+0000","last_clean":"2020-11-20T13:55:50.687223+0000","last_became_active":"2020-11-20T13:55:44.214492+0000","last_became_peered":"2020-11-20T13:55:44.214492+0000","last_unstale":"2020-11-20T13:55:50.687223+0000","last_undegraded":"2020-11-20T13:55:50.687223+0000","last_fullsized":"2020-11-20T13:55:50.687223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"24'8","reported_seq":"21","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.703742+0000","last_change":"2020-11-20T13:55:47.818676+0000","last_active":"2020-11-20T13:55:50.703742+0000","last_peered":"2020-11-20T13:55:50.703742+0000","last_clean":"2020-11-20T13:55:50.703742+0000","last_became_active":"2020-11-20T13:55:47.818355+0000","last_became_peered":"2020-11-20T13:55:47.818355+0000","last_unstale":"2020-11-20T13:55:50.703742+0000","last_undegraded":"2020-11-20T13:55:50.703742+0000","last_fullsized":"2020-11-20T13:55:50.703742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.693259+0000","last_change":"2020-11-20T13:55:50.691886+0000","last_active":"2020-11-20T13:55:50.693259+0000","last_peered":"2020-11-20T13:55:50.693259+0000","last_clean":"2020-11-20T13:55:50.693259+0000","last_became_active":"2020-11-20T13:55:50.691328+0000","last_became_peered":"2020-11-20T13:55:50.691328+0000","last_unstale":"2020-11-20T13:55:50.693259+0000","last_undegraded":"2020-11-20T13:55:50.693259+0000","last_fullsized":"2020-11-20T13:55:50.693259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"24'7","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.694810+0000","last_change":"2020-11-20T13:55:47.821416+0000","last_active":"2020-11-20T13:55:50.694810+0000","last_peered":"2020-11-20T13:55:50.694810+0000","last_clean":"2020-11-20T13:55:50.694810+0000","last_became_active":"2020-11-20T13:55:47.821069+0000","last_became_peered":"2020-11-20T13:55:47.821069+0000","last_unstale":"2020-11-20T13:55:50.694810+0000","last_undegraded":"2020-11-20T13:55:50.694810+0000","last_fullsized":"2020-11-20T13:55:50.694810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.687273+0000","last_change":"2020-11-20T13:55:44.202395+0000","last_active":"2020-11-20T13:55:50.687273+0000","last_peered":"2020-11-20T13:55:50.687273+0000","last_clean":"2020-11-20T13:55:50.687273+0000","last_became_active":"2020-11-20T13:55:44.201839+0000","last_became_peered":"2020-11-20T13:55:44.201839+0000","last_unstale":"2020-11-20T13:55:50.687273+0000","last_undegraded":"2020-11-20T13:55:50.687273+0000","last_fullsized":"2020-11-20T13:55:50.687273+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"12","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.699977+0000","last_change":"2020-11-20T13:55:50.694156+0000","last_active":"2020-11-20T13:55:50.699977+0000","last_peered":"2020-11-20T13:55:50.699977+0000","last_clean":"2020-11-20T13:55:50.699977+0000","last_became_active":"2020-11-20T13:55:50.693499+0000","last_became_peered":"2020-11-20T13:55:50.693499+0000","last_unstale":"2020-11-20T13:55:50.699977+0000","last_undegraded":"2020-11-20T13:55:50.699977+0000","last_fullsized":"2020-11-20T13:55:50.699977+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:49.315411+0000","last_clean_scrub_stamp":"2020-11-20T13:55:49.315411+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-11-20T13:55:50.692825+0000","last_change":"2020-11-20T13:55:44.201759+0000","last_active":"2020-11-20T13:55:50.692825+0000","last_peered":"2020-11-20T13:55:50.692825+0000","last_clean":"2020-11-20T13:55:50.692825+0000","last_became_active":"2020-11-20T13:55:44.201419+0000","last_became_peered":"2020-11-20T13:55:44.201419+0000","last_unstale":"2020-11-20T13:55:50.692825+0000","last_undegraded":"2020-11-20T13:55:50.692825+0000","last_fullsized":"2020-11-20T13:55:50.692825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:43.180862+0000","last_clean_scrub_stamp":"2020-11-20T13:55:43.180862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"25'4","reported_seq":"17","reported_epoch":"25","state":"active+clean","last_fresh":"2020-11-20T13:55:51.898189+0000","last_change":"2020-11-20T13:55:47.323565+0000","last_active":"2020-11-20T13:55:51.898189+0000","last_peered":"2020-11-20T13:55:51.898189+0000","last_clean":"2020-11-20T13:55:51.898189+0000","last_became_active":"2020-11-20T13:55:47.323216+0000","last_became_peered":"2020-11-20T13:55:47.323216+0000","last_unstale":"2020-11-20T13:55:51.898189+0000","last_undegraded":"2020-11-20T13:55:51.898189+0000","last_fullsized":"2020-11-20T13:55:51.898189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T13:55:46.298094+0000","last_clean_scrub_stamp":"2020-11-20T13:55:46.298094+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":109133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048844,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322996,"statfs":{"total":96636764160,"available":95562747904,"internally_reserved":1073741824,"allocated":266240,"data_stored":108558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":108592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048856,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322984,"statfs":{"total":96636764160,"available":95562735616,"internally_reserved":1073741824,"allocated":278528,"data_stored":110093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048860,"kb_used_data":276,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322980,"statfs":{"total":96636764160,"available":95562731520,"internally_reserved":1073741824,"allocated":282624,"data_stored":109293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048848,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322992,"statfs":{"total":96636764160,"available":95562743808,"internally_reserved":1073741824,"allocated":270336,"data_stored":108592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048852,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322988,"statfs":{"total":96636764160,"available":95562739712,"internally_reserved":1073741824,"allocated":274432,"data_stored":108718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]}]}]}} 2020-11-20T13:55:59.867 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-20T13:55:59.867 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-20T13:55:59.867 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-20T13:55:59.868 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-11-20T13:55:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.971+0000 7f2d4c31c700 1 Processor -- start 2020-11-20T13:55:59.972 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.971+0000 7f2d4c31c700 1 -- start start 2020-11-20T13:55:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4c31c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d44143d40 0x7f2d440aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4c31c700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d441441d0 con 0x7f2d441438e0 2020-11-20T13:55:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4c31c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4409b920 con 0x7f2d44098e30 2020-11-20T13:55:59.975 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4c31c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d440b5510 con 0x7f2d44143d40 2020-11-20T13:55:59.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4a0b8700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2d44098e30 0x7f2d4409b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59676/0 (socket says 172.21.15.117:59676) 2020-11-20T13:55:59.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4a0b8700 1 -- 172.21.15.117:0/3886364818 learned_addr learned my addr 172.21.15.117:0/3886364818 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:55:59.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4a8b9700 1 --2- 172.21.15.117:0/3886364818 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d44143d40 0x7f2d440aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204556526 0 0) 0x7f2d4409b920 con 0x7f2d44098e30 2020-11-20T13:55:59.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d2c003230 con 0x7f2d44098e30 2020-11-20T13:55:59.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804614999 0 0) 0x7f2d340010d0 con 0x7f2d441438e0 2020-11-20T13:55:59.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4409b920 con 0x7f2d441438e0 2020-11-20T13:55:59.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4a8b9700 1 -- 172.21.15.117:0/3886364818 >> v1:172.21.15.117:6789/0 conn(0x7f2d44098e30 legacy=0x7f2d4409b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 952501809 0 0) 0x7f2d2c003230 con 0x7f2d44098e30 2020-11-20T13:55:59.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4a8b9700 1 -- 172.21.15.117:0/3886364818 >> v1:172.21.15.85:6789/0 conn(0x7f2d441438e0 legacy=0x7f2d440ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4a8b9700 1 -- 172.21.15.117:0/3886364818 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d440b57b0 con 0x7f2d44143d40 2020-11-20T13:55:59.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.972+0000 7f2d4a8b9700 1 --2- 172.21.15.117:0/3886364818 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d44143d40 0x7f2d440aee00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2d40001110 tx=0x7f2d3c0091a0).ready entity=mon.2 client_cookie=735ad8b155273661 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d40001710 con 0x7f2d44143d40 2020-11-20T13:55:59.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c00b880 con 0x7f2d44143d40 2020-11-20T13:55:59.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d490b6700 1 -- 172.21.15.117:0/3886364818 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d3c00b9f0 con 0x7f2d44143d40 2020-11-20T13:55:59.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3886364818 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d44143d40 msgr2=0x7f2d440aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3886364818 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d44143d40 0x7f2d440aee00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2d40001110 tx=0x7f2d3c0091a0).stop 2020-11-20T13:55:59.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3886364818 shutdown_connections 2020-11-20T13:55:59.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3886364818 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d44143d40 0x7f2d440aee00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3886364818 >> 172.21.15.117:0/3886364818 conn(0x7f2d4405b9f0 msgr2=0x7f2d4405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:55:59.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3886364818 shutdown_connections 2020-11-20T13:55:59.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.973+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3886364818 wait complete. 2020-11-20T13:55:59.982 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.974+0000 7f2d4c31c700 1 Processor -- start 2020-11-20T13:55:59.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.974+0000 7f2d4c31c700 1 -- start start 2020-11-20T13:55:59.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4c31c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2d44098e30 0x7f2d4409f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4c31c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d441438e0 0x7f2d44159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4c31c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d44143d40 0x7f2d44066d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4c31c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4415a0b0 con 0x7f2d44098e30 2020-11-20T13:55:59.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4c31c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4415a350 con 0x7f2d44143d40 2020-11-20T13:55:59.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4c31c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4415a5f0 con 0x7f2d441438e0 2020-11-20T13:55:59.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d44143d40 0x7f2d44066d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d44143d40 0x7f2d44066d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40526/0 (socket says 172.21.15.117:40526) 2020-11-20T13:55:59.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 -- 172.21.15.117:0/3065509486 learned_addr learned my addr 172.21.15.117:0/3065509486 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:55:59.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a0b8700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2d44098e30 0x7f2d4409f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d498b7700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d441438e0 0x7f2d44159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 -- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d441438e0 msgr2=0x7f2d44159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d441438e0 0x7f2d44159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 -- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2d44098e30 msgr2=0x7f2d4409f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:55:59.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2d44098e30 0x7f2d4409f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:55:59.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 -- 172.21.15.117:0/3065509486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44067250 con 0x7f2d44143d40 2020-11-20T13:55:59.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d4a8b9700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d44143d40 0x7f2d44066d60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f2d40001110 tx=0x7f2d3c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.975+0000 7f2d337fe700 1 -- 172.21.15.117:0/3065509486 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d3c011b40 con 0x7f2d44143d40 2020-11-20T13:55:59.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.976+0000 7f2d337fe700 1 -- 172.21.15.117:0/3065509486 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c011ca0 con 0x7f2d44143d40 2020-11-20T13:55:59.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.976+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4413f370 con 0x7f2d44143d40 2020-11-20T13:55:59.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.976+0000 7f2d337fe700 1 -- 172.21.15.117:0/3065509486 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d3c020400 con 0x7f2d44143d40 2020-11-20T13:55:59.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.976+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4413f7e0 con 0x7f2d44143d40 2020-11-20T13:55:59.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.977+0000 7f2d337fe700 1 -- 172.21.15.117:0/3065509486 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2d3c027030 con 0x7f2d44143d40 2020-11-20T13:55:59.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.977+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d44159f00 con 0x7f2d44143d40 2020-11-20T13:55:59.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.977+0000 7f2d337fe700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2d24051290 0x7f2d24053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:55:59.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.977+0000 7f2d337fe700 1 -- 172.21.15.117:0/3065509486 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2d3c069cd0 con 0x7f2d44143d40 2020-11-20T13:55:59.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.977+0000 7f2d4a0b8700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2d24051290 0x7f2d24053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:55:59.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.977+0000 7f2d4a0b8700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2d24051290 0x7f2d24053750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2d4000d000 tx=0x7f2d40000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:55:59.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:55:59.980+0000 7f2d337fe700 1 -- 172.21.15.117:0/3065509486 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2d3c041ef0 con 0x7f2d44143d40 2020-11-20T13:56:00.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.188+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2d4415a7b0 con 0x7f2d44143d40 2020-11-20T13:56:00.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.188+0000 7f2d337fe700 1 -- 172.21.15.117:0/3065509486 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f2d3c025040 con 0x7f2d44143d40 2020-11-20T13:56:00.315 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2d24051290 msgr2=0x7f2d24053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2d24051290 0x7f2d24053750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2d4000d000 tx=0x7f2d40000b80).stop 2020-11-20T13:56:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d44143d40 msgr2=0x7f2d44066d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d44143d40 0x7f2d44066d60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f2d40001110 tx=0x7f2d3c016000).stop 2020-11-20T13:56:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 shutdown_connections 2020-11-20T13:56:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2d24051290 0x7f2d24053750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d44143d40 0x7f2d44066d60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2d44098e30 0x7f2d4409f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 --2- 172.21.15.117:0/3065509486 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2d441438e0 0x7f2d44159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 >> 172.21.15.117:0/3065509486 conn(0x7f2d4405b9f0 msgr2=0x7f2d4409bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 shutdown_connections 2020-11-20T13:56:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.190+0000 7f2d4c31c700 1 -- 172.21.15.117:0/3065509486 wait complete. 2020-11-20T13:56:00.321 INFO:teuthology.orchestra.run.smithi117.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-20T13:56:00.321 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-20T13:56:00.322 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.428+0000 7fd1d6d93700 1 Processor -- start 2020-11-20T13:56:00.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.428+0000 7fd1d6d93700 1 -- start start 2020-11-20T13:56:00.431 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6d93700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd1d009d3e0 0x7fd1d00b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6d93700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d009cd60 con 0x7fd1d005f130 2020-11-20T13:56:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6d93700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d00b4430 con 0x7fd1d005fb70 2020-11-20T13:56:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6d93700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d005f510 con 0x7fd1d009d3e0 2020-11-20T13:56:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d5590700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd1d009d3e0 0x7fd1d00b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6592700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd1d005fb70 0x7fd1d014aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59694/0 (socket says 172.21.15.117:59694) 2020-11-20T13:56:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d5590700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd1d009d3e0 0x7fd1d00b8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40532/0 (socket says 172.21.15.117:40532) 2020-11-20T13:56:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6592700 1 -- 172.21.15.117:0/4005503330 learned_addr learned my addr 172.21.15.117:0/4005503330 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d5590700 1 -- 172.21.15.117:0/4005503330 >> v1:172.21.15.117:6789/0 conn(0x7fd1d005fb70 legacy=0x7fd1d014aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d5590700 1 -- 172.21.15.117:0/4005503330 >> v1:172.21.15.85:6789/0 conn(0x7fd1d005f130 legacy=0x7fd1d01468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d4d8f700 1 -- 172.21.15.117:0/4005503330 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865233290 0 0) 0x7fd1d00b4430 con 0x7fd1d005fb70 2020-11-20T13:56:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d5590700 1 -- 172.21.15.117:0/4005503330 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00b46d0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d5590700 1 --2- 172.21.15.117:0/4005503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d00b8240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd1d00b7f40 tx=0x7fd1c0006dc0).ready entity=mon.1 client_cookie=76871a78659b1228 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d4d8f700 1 -- 172.21.15.117:0/4005503330 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1c00097b0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d4d8f700 1 -- 172.21.15.117:0/4005503330 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c0009910 con 0x7fd1d009d3e0 2020-11-20T13:56:00.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d4d8f700 1 -- 172.21.15.117:0/4005503330 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1c0009c70 con 0x7fd1d009d3e0 2020-11-20T13:56:00.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6d93700 1 -- 172.21.15.117:0/4005503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 msgr2=0x7fd1d00b8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.429+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/4005503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d00b8240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd1d00b7f40 tx=0x7fd1c0006dc0).stop 2020-11-20T13:56:00.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.430+0000 7fd1d6d93700 1 -- 172.21.15.117:0/4005503330 shutdown_connections 2020-11-20T13:56:00.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.430+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/4005503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d00b8240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.430+0000 7fd1d6d93700 1 -- 172.21.15.117:0/4005503330 >> 172.21.15.117:0/4005503330 conn(0x7fd1d00ba340 msgr2=0x7fd1d00ba750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:00.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.430+0000 7fd1d6d93700 1 -- 172.21.15.117:0/4005503330 shutdown_connections 2020-11-20T13:56:00.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.430+0000 7fd1d6d93700 1 -- 172.21.15.117:0/4005503330 wait complete. 2020-11-20T13:56:00.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.431+0000 7fd1d6d93700 1 Processor -- start 2020-11-20T13:56:00.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.431+0000 7fd1d6d93700 1 -- start start 2020-11-20T13:56:00.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd1d005f130 0x7fd1d00b7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd1d005fb70 0x7fd1d00b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d014b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d014b570 con 0x7fd1d005f130 2020-11-20T13:56:00.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01448c0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d014b790 con 0x7fd1d005fb70 2020-11-20T13:56:00.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d014b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d014b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40540/0 (socket says 172.21.15.117:40540) 2020-11-20T13:56:00.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 -- 172.21.15.117:0/371699197 learned_addr learned my addr 172.21.15.117:0/371699197 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:00.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d5d91700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd1d005f130 0x7fd1d00b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d5590700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd1d005fb70 0x7fd1d00b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 -- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd1d005fb70 msgr2=0x7fd1d00b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd1d005fb70 0x7fd1d00b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 -- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd1d005f130 msgr2=0x7fd1d00b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd1d005f130 0x7fd1d00b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 -- 172.21.15.117:0/371699197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d014ba30 con 0x7fd1d009d3e0 2020-11-20T13:56:00.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6592700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d014b080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd1c8000f00 tx=0x7fd1c8007cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:00.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1ceffd700 1 -- 172.21.15.117:0/371699197 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1c8005960 con 0x7fd1d009d3e0 2020-11-20T13:56:00.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1ceffd700 1 -- 172.21.15.117:0/371699197 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c8005ac0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d014bcd0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1ceffd700 1 -- 172.21.15.117:0/371699197 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1c8005e20 con 0x7fd1d009d3e0 2020-11-20T13:56:00.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.432+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d01468d0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.433+0000 7fd1ceffd700 1 -- 172.21.15.117:0/371699197 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd1c8010030 con 0x7fd1d009d3e0 2020-11-20T13:56:00.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.433+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d0057050 con 0x7fd1d009d3e0 2020-11-20T13:56:00.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.434+0000 7fd1ceffd700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd1b0051210 0x7fd1b00536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.434+0000 7fd1ceffd700 1 -- 172.21.15.117:0/371699197 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd1c8068fc0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.434+0000 7fd1d5d91700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd1b0051210 0x7fd1b00536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.434+0000 7fd1d5d91700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd1b0051210 0x7fd1b00536d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd1c400d000 tx=0x7fd1c4007ab0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:00.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.437+0000 7fd1ceffd700 1 -- 172.21.15.117:0/371699197 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd1c80411e0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.587 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.586+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1d00b94b0 con 0x7fd1d009d3e0 2020-11-20T13:56:00.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.587+0000 7fd1ceffd700 1 -- 172.21.15.117:0/371699197 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fd1c8044800 con 0x7fd1d009d3e0 2020-11-20T13:56:00.588 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:00.589 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:00.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd1b0051210 msgr2=0x7fd1b00536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd1b0051210 0x7fd1b00536d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fd1c400d000 tx=0x7fd1c4007ab0).stop 2020-11-20T13:56:00.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 msgr2=0x7fd1d014b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d014b080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd1c8000f00 tx=0x7fd1c8007cb0).stop 2020-11-20T13:56:00.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 shutdown_connections 2020-11-20T13:56:00.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd1b0051210 0x7fd1b00536d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d009d3e0 0x7fd1d014b080 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd1d005f130 0x7fd1d00b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 --2- 172.21.15.117:0/371699197 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd1d005fb70 0x7fd1d00b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 >> 172.21.15.117:0/371699197 conn(0x7fd1d00ba340 msgr2=0x7fd1d00b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:00.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 shutdown_connections 2020-11-20T13:56:00.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.588+0000 7fd1d6d93700 1 -- 172.21.15.117:0/371699197 wait complete. 2020-11-20T13:56:00.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:00.601 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:00.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.710+0000 7f40db7c3700 1 Processor -- start 2020-11-20T13:56:00.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.711+0000 7f40db7c3700 1 -- start start 2020-11-20T13:56:00.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.711+0000 7f40db7c3700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40d409cb40 0x7f40d41437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.711+0000 7f40db7c3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d4143c90 0x7f40d40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.711+0000 7f40db7c3700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40999d0 con 0x7f40d409bd80 2020-11-20T13:56:00.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.711+0000 7f40db7c3700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40997f0 con 0x7f40d409cb40 2020-11-20T13:56:00.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.711+0000 7f40db7c3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4099c70 con 0x7f40d4143c90 2020-11-20T13:56:00.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.711+0000 7f40d8d5e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40d409cb40 0x7f40d41437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d8d5e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40d409cb40 0x7f40d41437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40544/0 (socket says 172.21.15.117:40544) 2020-11-20T13:56:00.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d8d5e700 1 -- 172.21.15.117:0/3854633517 learned_addr learned my addr 172.21.15.117:0/3854633517 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:00.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d9d60700 1 --2- 172.21.15.117:0/3854633517 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d4143c90 0x7f40d40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d8d5e700 1 -- 172.21.15.117:0/3854633517 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d4143c90 msgr2=0x7f40d40b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:00.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d8d5e700 1 --2- 172.21.15.117:0/3854633517 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d4143c90 0x7f40d40b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d8d5e700 1 -- 172.21.15.117:0/3854633517 >> v1:172.21.15.85:6789/0 conn(0x7f40d409bd80 legacy=0x7f40d409a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d8d5e700 1 -- 172.21.15.117:0/3854633517 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4144120 con 0x7f40d409cb40 2020-11-20T13:56:00.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d9d60700 1 --2- 172.21.15.117:0/3854633517 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d4143c90 0x7f40d40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40d8d5e700 1 --2- 172.21.15.117:0/3854633517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d409cb40 0x7f40d41437a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f40d0002f30 tx=0x7f40d0006e80).ready entity=mon.1 client_cookie=2f04b025fd7f1083 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40c3fff700 1 -- 172.21.15.117:0/3854633517 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40d00097f0 con 0x7f40d409cb40 2020-11-20T13:56:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40c3fff700 1 -- 172.21.15.117:0/3854633517 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0009950 con 0x7f40d409cb40 2020-11-20T13:56:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40c3fff700 1 -- 172.21.15.117:0/3854633517 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40d0009cb0 con 0x7f40d409cb40 2020-11-20T13:56:00.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40db7c3700 1 -- 172.21.15.117:0/3854633517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d409cb40 msgr2=0x7f40d41437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.712+0000 7f40db7c3700 1 --2- 172.21.15.117:0/3854633517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d409cb40 0x7f40d41437a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f40d0002f30 tx=0x7f40d0006e80).stop 2020-11-20T13:56:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.713+0000 7f40db7c3700 1 -- 172.21.15.117:0/3854633517 shutdown_connections 2020-11-20T13:56:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.713+0000 7f40db7c3700 1 --2- 172.21.15.117:0/3854633517 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d4143c90 0x7f40d40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.713+0000 7f40db7c3700 1 --2- 172.21.15.117:0/3854633517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d409cb40 0x7f40d41437a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.713+0000 7f40db7c3700 1 -- 172.21.15.117:0/3854633517 >> 172.21.15.117:0/3854633517 conn(0x7f40d40b9770 msgr2=0x7f40d40b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.713+0000 7f40db7c3700 1 -- 172.21.15.117:0/3854633517 shutdown_connections 2020-11-20T13:56:00.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.713+0000 7f40db7c3700 1 -- 172.21.15.117:0/3854633517 wait complete. 2020-11-20T13:56:00.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 Processor -- start 2020-11-20T13:56:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 -- start start 2020-11-20T13:56:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d409bd80 0x7f40d4159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40d409cb40 0x7f40d40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d4143c90 0x7f40d41417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4156a70 con 0x7f40d409cb40 2020-11-20T13:56:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4156d10 con 0x7f40d4143c90 2020-11-20T13:56:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40db7c3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4156fb0 con 0x7f40d409bd80 2020-11-20T13:56:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40d9d60700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d4143c90 0x7f40d41417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40d9d60700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d4143c90 0x7f40d41417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40552/0 (socket says 172.21.15.117:40552) 2020-11-20T13:56:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40d9d60700 1 -- 172.21.15.117:0/1050753490 learned_addr learned my addr 172.21.15.117:0/1050753490 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40d955f700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d409bd80 0x7f40d4159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.714+0000 7f40d8d5e700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40d409cb40 0x7f40d40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40d9d60700 1 -- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d409bd80 msgr2=0x7f40d4159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40d9d60700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d409bd80 0x7f40d4159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40d9d60700 1 -- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40d409cb40 msgr2=0x7f40d40b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40d9d60700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40d409cb40 0x7f40d40b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40d9d60700 1 -- 172.21.15.117:0/1050753490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4065d90 con 0x7f40d4143c90 2020-11-20T13:56:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40d9d60700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d4143c90 0x7f40d41417b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f40cc006dd0 tx=0x7f40cc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40c27fc700 1 -- 172.21.15.117:0/1050753490 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40cc00c810 con 0x7f40d4143c90 2020-11-20T13:56:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40c27fc700 1 -- 172.21.15.117:0/1050753490 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc00c970 con 0x7f40d4143c90 2020-11-20T13:56:00.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d4141ca0 con 0x7f40d4143c90 2020-11-20T13:56:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40c27fc700 1 -- 172.21.15.117:0/1050753490 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40cc00ccd0 con 0x7f40d4143c90 2020-11-20T13:56:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.715+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d4142190 con 0x7f40d4143c90 2020-11-20T13:56:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.716+0000 7f40c27fc700 1 -- 172.21.15.117:0/1050753490 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f40cc01d460 con 0x7f40d4143c90 2020-11-20T13:56:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.716+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d4157170 con 0x7f40d4143c90 2020-11-20T13:56:00.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.716+0000 7f40c27fc700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40b4051290 0x7f40b4053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.716+0000 7f40c27fc700 1 -- 172.21.15.117:0/1050753490 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f40cc069620 con 0x7f40d4143c90 2020-11-20T13:56:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.716+0000 7f40d955f700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40b4051290 0x7f40b4053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.717+0000 7f40d955f700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40b4051290 0x7f40b4053750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f40c400d000 tx=0x7f40c4000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:00.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.720+0000 7f40c27fc700 1 -- 172.21.15.117:0/1050753490 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f40cc0416d0 con 0x7f40d4143c90 2020-11-20T13:56:00.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.869+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f40d4057050 con 0x7f40d4143c90 2020-11-20T13:56:00.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.870+0000 7f40c27fc700 1 -- 172.21.15.117:0/1050753490 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f40cc044cf0 con 0x7f40d4143c90 2020-11-20T13:56:00.871 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:00.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40b4051290 msgr2=0x7f40b4053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40b4051290 0x7f40b4053750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f40c400d000 tx=0x7f40c4000b80).stop 2020-11-20T13:56:00.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d4143c90 msgr2=0x7f40d41417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d4143c90 0x7f40d41417b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f40cc006dd0 tx=0x7f40cc008040).stop 2020-11-20T13:56:00.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 shutdown_connections 2020-11-20T13:56:00.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40b4051290 0x7f40b4053750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40d409bd80 0x7f40d4159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40d4143c90 0x7f40d41417b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 --2- 172.21.15.117:0/1050753490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40d409cb40 0x7f40d40b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 >> 172.21.15.117:0/1050753490 conn(0x7f40d40b9770 msgr2=0x7f40d4098b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:00.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 shutdown_connections 2020-11-20T13:56:00.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.871+0000 7f40db7c3700 1 -- 172.21.15.117:0/1050753490 wait complete. 2020-11-20T13:56:00.883 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:00.992 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.990+0000 7f6ef888f700 1 Processor -- start 2020-11-20T13:56:00.992 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.990+0000 7f6ef888f700 1 -- start start 2020-11-20T13:56:00.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.991+0000 7f6ef888f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ef4143eb0 0x7f6ef40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:00.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.991+0000 7f6ef888f700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40999d0 con 0x7f6ef409cb40 2020-11-20T13:56:00.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.991+0000 7f6ef888f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef4099c70 con 0x7f6ef409bd80 2020-11-20T13:56:00.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.991+0000 7f6ef888f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef40997f0 con 0x7f6ef4143eb0 2020-11-20T13:56:00.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.991+0000 7f6ef2d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ef4143eb0 0x7f6ef40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:00.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.991+0000 7f6ef259c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6ef409bd80 0x7f6ef409a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59716/0 (socket says 172.21.15.117:59716) 2020-11-20T13:56:00.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.991+0000 7f6ef259c700 1 -- 172.21.15.117:0/771667482 learned_addr learned my addr 172.21.15.117:0/771667482 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:00.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef2d9d700 1 -- 172.21.15.117:0/771667482 >> v1:172.21.15.117:6789/0 conn(0x7f6ef409bd80 legacy=0x7f6ef409a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef2d9d700 1 -- 172.21.15.117:0/771667482 >> v1:172.21.15.85:6789/0 conn(0x7f6ef409cb40 legacy=0x7f6ef41437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:00.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef1d9b700 1 -- 172.21.15.117:0/771667482 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977826142 0 0) 0x7f6ef4099c70 con 0x7f6ef409bd80 2020-11-20T13:56:00.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef2d9d700 1 -- 172.21.15.117:0/771667482 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef409c160 con 0x7f6ef4143eb0 2020-11-20T13:56:00.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef2d9d700 1 --2- 172.21.15.117:0/771667482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef4143eb0 0x7f6ef40b4110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6ef409ad40 tx=0x7f6eec0091a0).ready entity=mon.1 client_cookie=4baa64b390ca4a99 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:00.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef1d9b700 1 -- 172.21.15.117:0/771667482 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eec00ba10 con 0x7f6ef4143eb0 2020-11-20T13:56:00.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef1d9b700 1 -- 172.21.15.117:0/771667482 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec00bb70 con 0x7f6ef4143eb0 2020-11-20T13:56:00.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef1d9b700 1 -- 172.21.15.117:0/771667482 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6eec00bce0 con 0x7f6ef4143eb0 2020-11-20T13:56:00.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef888f700 1 -- 172.21.15.117:0/771667482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef4143eb0 msgr2=0x7f6ef40b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:00.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef888f700 1 --2- 172.21.15.117:0/771667482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef4143eb0 0x7f6ef40b4110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6ef409ad40 tx=0x7f6eec0091a0).stop 2020-11-20T13:56:00.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef888f700 1 -- 172.21.15.117:0/771667482 shutdown_connections 2020-11-20T13:56:00.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef888f700 1 --2- 172.21.15.117:0/771667482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef4143eb0 0x7f6ef40b4110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:00.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef888f700 1 -- 172.21.15.117:0/771667482 >> 172.21.15.117:0/771667482 conn(0x7f6ef40b9770 msgr2=0x7f6ef40b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:00.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef888f700 1 -- 172.21.15.117:0/771667482 shutdown_connections 2020-11-20T13:56:00.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.992+0000 7f6ef888f700 1 -- 172.21.15.117:0/771667482 wait complete. 2020-11-20T13:56:01.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.993+0000 7f6ef888f700 1 Processor -- start 2020-11-20T13:56:01.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.993+0000 7f6ef888f700 1 -- start start 2020-11-20T13:56:01.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef888f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef409bd80 0x7f6ef4144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef888f700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6ef409cb40 0x7f6ef4148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef888f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6ef4143eb0 0x7f6ef4066f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef888f700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4067450 con 0x7f6ef409cb40 2020-11-20T13:56:01.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef888f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4067670 con 0x7f6ef409bd80 2020-11-20T13:56:01.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef888f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4157810 con 0x7f6ef4143eb0 2020-11-20T13:56:01.003 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef409bd80 0x7f6ef4144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.003 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef409bd80 0x7f6ef4144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40562/0 (socket says 172.21.15.117:40562) 2020-11-20T13:56:01.003 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 -- 172.21.15.117:0/1162155489 learned_addr learned my addr 172.21.15.117:0/1162155489 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:01.003 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 -- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6ef4143eb0 msgr2=0x7f6ef4066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.003 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6eebfff700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6ef409cb40 0x7f6ef4148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.004 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef2d9d700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6ef4143eb0 0x7f6ef4066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.004 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6ef4143eb0 0x7f6ef4066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.004 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 -- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6ef409cb40 msgr2=0x7f6ef4148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.004 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6ef409cb40 0x7f6ef4148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.004 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 -- 172.21.15.117:0/1162155489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4157ab0 con 0x7f6ef409bd80 2020-11-20T13:56:01.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.994+0000 7f6ef259c700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef409bd80 0x7f6ef4144320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f6ef4154230 tx=0x7f6ee40079b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.995+0000 7f6eeb7fe700 1 -- 172.21.15.117:0/1162155489 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee40072e0 con 0x7f6ef409bd80 2020-11-20T13:56:01.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.995+0000 7f6eeb7fe700 1 -- 172.21.15.117:0/1162155489 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee4007440 con 0x7f6ef409bd80 2020-11-20T13:56:01.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.995+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef4157f50 con 0x7f6ef409bd80 2020-11-20T13:56:01.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.995+0000 7f6eeb7fe700 1 -- 172.21.15.117:0/1162155489 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee40057d0 con 0x7f6ef409bd80 2020-11-20T13:56:01.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.995+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef41584f0 con 0x7f6ef409bd80 2020-11-20T13:56:01.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.996+0000 7f6eeb7fe700 1 -- 172.21.15.117:0/1162155489 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f6ee4010070 con 0x7f6ef409bd80 2020-11-20T13:56:01.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.996+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef4057050 con 0x7f6ef409bd80 2020-11-20T13:56:01.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.996+0000 7f6eeb7fe700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6ed80513c0 0x7f6ed8053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.007 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.996+0000 7f6eeb7fe700 1 -- 172.21.15.117:0/1162155489 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f6ee4068860 con 0x7f6ef409bd80 2020-11-20T13:56:01.007 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.996+0000 7f6eebfff700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6ed80513c0 0x7f6ed8053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.007 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.996+0000 7f6eebfff700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6ed80513c0 0x7f6ed8053880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f6edc008000 tx=0x7f6edc009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:00.999+0000 7f6eeb7fe700 1 -- 172.21.15.117:0/1162155489 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6ee4040980 con 0x7f6ef409bd80 2020-11-20T13:56:01.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.149+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6ef4157c40 con 0x7f6ef409bd80 2020-11-20T13:56:01.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.150+0000 7f6eeb7fe700 1 -- 172.21.15.117:0/1162155489 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6ee4018070 con 0x7f6ef409bd80 2020-11-20T13:56:01.151 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:01.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6ed80513c0 msgr2=0x7f6ed8053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6ed80513c0 0x7f6ed8053880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f6edc008000 tx=0x7f6edc009040).stop 2020-11-20T13:56:01.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef409bd80 msgr2=0x7f6ef4144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef409bd80 0x7f6ef4144320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f6ef4154230 tx=0x7f6ee40079b0).stop 2020-11-20T13:56:01.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 shutdown_connections 2020-11-20T13:56:01.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6ed80513c0 0x7f6ed8053880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6ef4143eb0 0x7f6ef4066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef409bd80 0x7f6ef4144320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 --2- 172.21.15.117:0/1162155489 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6ef409cb40 0x7f6ef4148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 >> 172.21.15.117:0/1162155489 conn(0x7f6ef40b9770 msgr2=0x7f6ef40b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:01.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 shutdown_connections 2020-11-20T13:56:01.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.151+0000 7f6ef888f700 1 -- 172.21.15.117:0/1162155489 wait complete. 2020-11-20T13:56:01.163 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:01.271 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.270+0000 7ff70907b700 1 Processor -- start 2020-11-20T13:56:01.272 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.270+0000 7ff70907b700 1 -- start start 2020-11-20T13:56:01.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff70907b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff70409bd80 0x7ff70409cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff70907b700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7040999d0 con 0x7ff70409a110 2020-11-20T13:56:01.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff70907b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff704099c70 con 0x7ff704143eb0 2020-11-20T13:56:01.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff70907b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70409c160 con 0x7ff70409bd80 2020-11-20T13:56:01.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff702d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff70409bd80 0x7ff70409cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff70359e700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff704143eb0 0x7ff7040b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59730/0 (socket says 172.21.15.117:59730) 2020-11-20T13:56:01.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff702d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff70409bd80 0x7ff70409cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40570/0 (socket says 172.21.15.117:40570) 2020-11-20T13:56:01.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.271+0000 7ff70359e700 1 -- 172.21.15.117:0/3453897063 learned_addr learned my addr 172.21.15.117:0/3453897063 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff702d9d700 1 -- 172.21.15.117:0/3453897063 >> v1:172.21.15.117:6789/0 conn(0x7ff704143eb0 legacy=0x7ff7040b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff702d9d700 1 -- 172.21.15.117:0/3453897063 >> v1:172.21.15.85:6789/0 conn(0x7ff70409a110 legacy=0x7ff7041437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff702d9d700 1 -- 172.21.15.117:0/3453897063 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70409a5c0 con 0x7ff70409bd80 2020-11-20T13:56:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff702d9d700 1 --2- 172.21.15.117:0/3453897063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409bd80 0x7ff70409cb40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff6f8002f30 tx=0x7ff6f8006dc0).ready entity=mon.1 client_cookie=aa470d3b9452434 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff701d9b700 1 -- 172.21.15.117:0/3453897063 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f80097f0 con 0x7ff70409bd80 2020-11-20T13:56:01.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff701d9b700 1 -- 172.21.15.117:0/3453897063 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8009950 con 0x7ff70409bd80 2020-11-20T13:56:01.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff701d9b700 1 -- 172.21.15.117:0/3453897063 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f8009cb0 con 0x7ff70409bd80 2020-11-20T13:56:01.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff70907b700 1 -- 172.21.15.117:0/3453897063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409bd80 msgr2=0x7ff70409cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.272+0000 7ff70907b700 1 --2- 172.21.15.117:0/3453897063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409bd80 0x7ff70409cb40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff6f8002f30 tx=0x7ff6f8006dc0).stop 2020-11-20T13:56:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.273+0000 7ff70907b700 1 -- 172.21.15.117:0/3453897063 shutdown_connections 2020-11-20T13:56:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.273+0000 7ff70907b700 1 --2- 172.21.15.117:0/3453897063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409bd80 0x7ff70409cb40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.273+0000 7ff70907b700 1 -- 172.21.15.117:0/3453897063 >> 172.21.15.117:0/3453897063 conn(0x7ff7040b9770 msgr2=0x7ff7040b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.273+0000 7ff70907b700 1 -- 172.21.15.117:0/3453897063 shutdown_connections 2020-11-20T13:56:01.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.273+0000 7ff70907b700 1 -- 172.21.15.117:0/3453897063 wait complete. 2020-11-20T13:56:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.273+0000 7ff70907b700 1 Processor -- start 2020-11-20T13:56:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff70907b700 1 -- start start 2020-11-20T13:56:01.281 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff70907b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409a110 0x7ff70409f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff70907b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff70409bd80 0x7ff7040b8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff70907b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff704143eb0 0x7ff70413d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff70907b700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff704066d40 con 0x7ff70409bd80 2020-11-20T13:56:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff70907b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff704066fe0 con 0x7ff70409a110 2020-11-20T13:56:01.282 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff70907b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff704067280 con 0x7ff704143eb0 2020-11-20T13:56:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff702d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409a110 0x7ff70409f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff702d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409a110 0x7ff70409f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40572/0 (socket says 172.21.15.117:40572) 2020-11-20T13:56:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff702d9d700 1 -- 172.21.15.117:0/1680436342 learned_addr learned my addr 172.21.15.117:0/1680436342 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.274+0000 7ff702d9d700 1 -- 172.21.15.117:0/1680436342 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff704143eb0 msgr2=0x7ff70413d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff702d9d700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff704143eb0 0x7ff70413d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:01.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff702d9d700 1 -- 172.21.15.117:0/1680436342 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff70409bd80 msgr2=0x7ff7040b8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff702d9d700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff70409bd80 0x7ff7040b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:01.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff702d9d700 1 -- 172.21.15.117:0/1680436342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7040ae740 con 0x7ff70409a110 2020-11-20T13:56:01.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff702d9d700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409a110 0x7ff70409f2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7ff6f8002f30 tx=0x7ff6f8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff6f3fff700 1 -- 172.21.15.117:0/1680436342 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f801d450 con 0x7ff70409a110 2020-11-20T13:56:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff6f3fff700 1 -- 172.21.15.117:0/1680436342 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f801d5b0 con 0x7ff70409a110 2020-11-20T13:56:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff6f3fff700 1 -- 172.21.15.117:0/1680436342 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f801d910 con 0x7ff70409a110 2020-11-20T13:56:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70409ef10 con 0x7ff70409a110 2020-11-20T13:56:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.275+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7040678e0 con 0x7ff70409a110 2020-11-20T13:56:01.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.276+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7040ae8d0 con 0x7ff70409a110 2020-11-20T13:56:01.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.276+0000 7ff6f3fff700 1 -- 172.21.15.117:0/1680436342 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff6f8024030 con 0x7ff70409a110 2020-11-20T13:56:01.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.276+0000 7ff6f3fff700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff6e0051320 0x7ff6e00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.276+0000 7ff6f3fff700 1 -- 172.21.15.117:0/1680436342 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff6f806a4c0 con 0x7ff70409a110 2020-11-20T13:56:01.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.276+0000 7ff70259c700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff6e0051320 0x7ff6e00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.276+0000 7ff70259c700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff6e0051320 0x7ff6e00537e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7ff70409afe0 tx=0x7ff6f40077f0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.287 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.279+0000 7ff6f3fff700 1 -- 172.21.15.117:0/1680436342 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff6f80426e0 con 0x7ff70409a110 2020-11-20T13:56:01.430 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.429+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff704067440 con 0x7ff70409a110 2020-11-20T13:56:01.431 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.429+0000 7ff6f3fff700 1 -- 172.21.15.117:0/1680436342 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff6f8045d00 con 0x7ff70409a110 2020-11-20T13:56:01.431 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:01.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff6e0051320 msgr2=0x7ff6e00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff6e0051320 0x7ff6e00537e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7ff70409afe0 tx=0x7ff6f40077f0).stop 2020-11-20T13:56:01.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409a110 msgr2=0x7ff70409f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409a110 0x7ff70409f2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7ff6f8002f30 tx=0x7ff6f8015000).stop 2020-11-20T13:56:01.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 shutdown_connections 2020-11-20T13:56:01.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff6e0051320 0x7ff6e00537e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff704143eb0 0x7ff70413d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff70409a110 0x7ff70409f2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 --2- 172.21.15.117:0/1680436342 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff70409bd80 0x7ff7040b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 >> 172.21.15.117:0/1680436342 conn(0x7ff7040b9770 msgr2=0x7ff704062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:01.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 shutdown_connections 2020-11-20T13:56:01.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.431+0000 7ff70907b700 1 -- 172.21.15.117:0/1680436342 wait complete. 2020-11-20T13:56:01.442 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:01.549 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.548+0000 7fadc72c2700 1 Processor -- start 2020-11-20T13:56:01.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.548+0000 7fadc72c2700 1 -- start start 2020-11-20T13:56:01.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc72c2700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fadc009bd80 0x7fadc009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc72c2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc01437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc72c2700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc00999d0 con 0x7fadc0143c90 2020-11-20T13:56:01.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc72c2700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc0099c70 con 0x7fadc009bd80 2020-11-20T13:56:01.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc72c2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0144120 con 0x7fadc009a110 2020-11-20T13:56:01.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc505e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fadc009bd80 0x7fadc009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc485d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc01437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc505e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fadc009bd80 0x7fadc009cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:52206/0 (socket says 172.21.15.117:52206) 2020-11-20T13:56:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc505e700 1 -- 172.21.15.117:0/673502056 learned_addr learned my addr 172.21.15.117:0/673502056 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.549+0000 7fadc585f700 1 --1- 172.21.15.117:0/673502056 >> v1:172.21.15.85:6789/0 conn(0x7fadc0143c90 0x7fadc00b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:43644/0 (socket says 172.21.15.117:43644) 2020-11-20T13:56:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadc505e700 1 -- 172.21.15.117:0/673502056 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 msgr2=0x7fadc01437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadc505e700 1 --2- 172.21.15.117:0/673502056 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc01437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadc505e700 1 -- 172.21.15.117:0/673502056 >> v1:172.21.15.85:6789/0 conn(0x7fadc0143c90 legacy=0x7fadc00b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadc505e700 1 -- 172.21.15.117:0/673502056 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc009c160 con 0x7fadc009bd80 2020-11-20T13:56:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadc505e700 1 --2- 172.21.15.117:0/673502056 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 0x7fadc009cb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fadc009b9c0 tx=0x7fadb0006e80).ready entity=mon.0 client_cookie=aee6621d72d0359f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadaffff700 1 -- 172.21.15.117:0/673502056 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb00097b0 con 0x7fadc009bd80 2020-11-20T13:56:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadaffff700 1 -- 172.21.15.117:0/673502056 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0009910 con 0x7fadc009bd80 2020-11-20T13:56:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadaffff700 1 -- 172.21.15.117:0/673502056 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb0009c70 con 0x7fadc009bd80 2020-11-20T13:56:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadc72c2700 1 -- 172.21.15.117:0/673502056 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 msgr2=0x7fadc009cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.550+0000 7fadc72c2700 1 --2- 172.21.15.117:0/673502056 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 0x7fadc009cb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fadc009b9c0 tx=0x7fadb0006e80).stop 2020-11-20T13:56:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.551+0000 7fadc72c2700 1 -- 172.21.15.117:0/673502056 shutdown_connections 2020-11-20T13:56:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.551+0000 7fadc72c2700 1 --2- 172.21.15.117:0/673502056 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 0x7fadc009cb40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.551+0000 7fadc72c2700 1 --2- 172.21.15.117:0/673502056 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc01437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.551+0000 7fadc72c2700 1 -- 172.21.15.117:0/673502056 >> 172.21.15.117:0/673502056 conn(0x7fadc00b9770 msgr2=0x7fadc00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:01.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.551+0000 7fadc72c2700 1 -- 172.21.15.117:0/673502056 shutdown_connections 2020-11-20T13:56:01.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.551+0000 7fadc72c2700 1 -- 172.21.15.117:0/673502056 wait complete. 2020-11-20T13:56:01.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 Processor -- start 2020-11-20T13:56:01.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 -- start start 2020-11-20T13:56:01.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc0067470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 0x7fadc0144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0143c90 0x7fadc0144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0157e60 con 0x7fadc009bd80 2020-11-20T13:56:01.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0158100 con 0x7fadc0143c90 2020-11-20T13:56:01.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc72c2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc01583a0 con 0x7fadc009a110 2020-11-20T13:56:01.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc585f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0143c90 0x7fadc0144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc505e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc0067470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc585f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0143c90 0x7fadc0144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40588/0 (socket says 172.21.15.117:40588) 2020-11-20T13:56:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc485d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 0x7fadc0144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.552+0000 7fadc585f700 1 -- 172.21.15.117:0/2577040785 learned_addr learned my addr 172.21.15.117:0/2577040785 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc585f700 1 -- 172.21.15.117:0/2577040785 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 msgr2=0x7fadc0067470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc585f700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc0067470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc585f700 1 -- 172.21.15.117:0/2577040785 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 msgr2=0x7fadc0144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc585f700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 0x7fadc0144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc585f700 1 -- 172.21.15.117:0/2577040785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc0157ae0 con 0x7fadc0143c90 2020-11-20T13:56:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc585f700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0143c90 0x7fadc0144750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fadc009ad40 tx=0x7fadb8007ea0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadae7fc700 1 -- 172.21.15.117:0/2577040785 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb8005ab0 con 0x7fadc0143c90 2020-11-20T13:56:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadae7fc700 1 -- 172.21.15.117:0/2577040785 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb8005c10 con 0x7fadc0143c90 2020-11-20T13:56:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadae7fc700 1 -- 172.21.15.117:0/2577040785 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadb8007110 con 0x7fadc0143c90 2020-11-20T13:56:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc0144c40 con 0x7fadc0143c90 2020-11-20T13:56:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.553+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc00b1990 con 0x7fadc0143c90 2020-11-20T13:56:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.554+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc0158560 con 0x7fadc0143c90 2020-11-20T13:56:01.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.554+0000 7fadae7fc700 1 -- 172.21.15.117:0/2577040785 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fadb8010030 con 0x7fadc0143c90 2020-11-20T13:56:01.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.554+0000 7fadae7fc700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fada0051320 0x7fada00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.554+0000 7fadae7fc700 1 -- 172.21.15.117:0/2577040785 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fadb8068f60 con 0x7fadc0143c90 2020-11-20T13:56:01.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.554+0000 7fadc505e700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fada0051320 0x7fada00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.555+0000 7fadc505e700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fada0051320 0x7fada00537e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fadb001c000 tx=0x7fadb0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.558+0000 7fadae7fc700 1 -- 172.21.15.117:0/2577040785 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fadb8041120 con 0x7fadc0143c90 2020-11-20T13:56:01.709 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.708+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadc0057050 con 0x7fadc0143c90 2020-11-20T13:56:01.710 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.708+0000 7fadae7fc700 1 -- 172.21.15.117:0/2577040785 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7fadb8044740 con 0x7fadc0143c90 2020-11-20T13:56:01.710 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:01.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fada0051320 msgr2=0x7fada00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fada0051320 0x7fada00537e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fadb001c000 tx=0x7fadb0015000).stop 2020-11-20T13:56:01.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0143c90 msgr2=0x7fadc0144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0143c90 0x7fadc0144750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fadc009ad40 tx=0x7fadb8007ea0).stop 2020-11-20T13:56:01.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 shutdown_connections 2020-11-20T13:56:01.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fada0051320 0x7fada00537e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fadc009a110 0x7fadc0067470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc0143c90 0x7fadc0144750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 --2- 172.21.15.117:0/2577040785 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fadc009bd80 0x7fadc0144260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.709+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 >> 172.21.15.117:0/2577040785 conn(0x7fadc00b9770 msgr2=0x7fadc0142b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:01.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.710+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 shutdown_connections 2020-11-20T13:56:01.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.710+0000 7fadc72c2700 1 -- 172.21.15.117:0/2577040785 wait complete. 2020-11-20T13:56:01.722 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:01.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.829+0000 7f9733958700 1 Processor -- start 2020-11-20T13:56:01.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.829+0000 7f9733958700 1 -- start start 2020-11-20T13:56:01.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f9733958700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f972c1438e0 0x7f972c143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f9733958700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 0x7f972c0aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f9733958700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972c09b920 con 0x7f972c098e30 2020-11-20T13:56:01.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f9733958700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972c144230 con 0x7f972c1438e0 2020-11-20T13:56:01.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f9733958700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c0b5510 con 0x7f972c0ae6f0 2020-11-20T13:56:01.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f97316f4700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f972c098e30 0x7f972c09b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59752/0 (socket says 172.21.15.117:59752) 2020-11-20T13:56:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f97316f4700 1 -- 172.21.15.117:0/3934054470 learned_addr learned my addr 172.21.15.117:0/3934054470 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:01.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f9730ef3700 1 --2- 172.21.15.117:0/3934054470 >> v2:172.21.15.85:3300/0 conn(0x7f972c1438e0 0x7f972c143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f9731ef5700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 0x7f972c0aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250118698 0 0) 0x7f972c09b920 con 0x7f972c098e30 2020-11-20T13:56:01.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9714003230 con 0x7f972c098e30 2020-11-20T13:56:01.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.830+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1289271455 0 0) 0x7f9714003230 con 0x7f972c098e30 2020-11-20T13:56:01.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 msgr2=0x7f972c0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:01.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 0x7f972c0aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 >> v2:172.21.15.85:3300/0 conn(0x7f972c1438e0 msgr2=0x7f972c143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:01.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 --2- 172.21.15.117:0/3934054470 >> v2:172.21.15.85:3300/0 conn(0x7f972c1438e0 0x7f972c143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c0b57b0 con 0x7f972c098e30 2020-11-20T13:56:01.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f971c0016d0 con 0x7f972c098e30 2020-11-20T13:56:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 >> v1:172.21.15.117:6789/0 conn(0x7f972c098e30 legacy=0x7f972c09b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9714003720 0x7f9714003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f97140044d0 0x7f9714004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 0x7f9714004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c144230 con 0x7f97140044d0 2020-11-20T13:56:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c0b5510 con 0x7f9714003720 2020-11-20T13:56:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9714004260 con 0x7f972c0ae6f0 2020-11-20T13:56:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f97316f4700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9714003720 0x7f9714003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f9730ef3700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f97140044d0 0x7f9714004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f9731ef5700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 0x7f9714004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f97316f4700 1 -- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 msgr2=0x7f9714004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f97316f4700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c0ae6f0 0x7f9714004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f97316f4700 1 -- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f97140044d0 msgr2=0x7f9714004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f97316f4700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f97140044d0 0x7f9714004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f97316f4700 1 -- 172.21.15.117:0/3934054470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714003230 con 0x7f9714003720 2020-11-20T13:56:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f97316f4700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9714003720 0x7f9714003b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f971c0013e0 tx=0x7f971c007740).ready entity=mon.1 client_cookie=e50f4295857f7ccf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f971c0070d0 con 0x7f9714003720 2020-11-20T13:56:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c007230 con 0x7f9714003720 2020-11-20T13:56:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f971bfff700 1 -- 172.21.15.117:0/3934054470 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f971c0055e0 con 0x7f9714003720 2020-11-20T13:56:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f9733958700 1 -- 172.21.15.117:0/3934054470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9714003720 msgr2=0x7f9714003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f9733958700 1 --2- 172.21.15.117:0/3934054470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9714003720 0x7f9714003b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f971c0013e0 tx=0x7f971c007740).stop 2020-11-20T13:56:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.831+0000 7f9731ef5700 1 -- 172.21.15.117:0/3934054470 reap_dead start 2020-11-20T13:56:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.832+0000 7f9733958700 1 -- 172.21.15.117:0/3934054470 shutdown_connections 2020-11-20T13:56:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.832+0000 7f9733958700 1 -- 172.21.15.117:0/3934054470 >> 172.21.15.117:0/3934054470 conn(0x7f972c05b9f0 msgr2=0x7f972c05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.832+0000 7f9733958700 1 -- 172.21.15.117:0/3934054470 shutdown_connections 2020-11-20T13:56:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.832+0000 7f9733958700 1 -- 172.21.15.117:0/3934054470 wait complete. 2020-11-20T13:56:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 Processor -- start 2020-11-20T13:56:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 -- start start 2020-11-20T13:56:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f972c0b2cc0 0x7f972c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c066d20 0x7f972c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c157d80 0x7f972c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c154ed0 con 0x7f972c0b2cc0 2020-11-20T13:56:01.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c0671c0 con 0x7f972c157d80 2020-11-20T13:56:01.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.833+0000 7f9733958700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c067460 con 0x7f972c066d20 2020-11-20T13:56:01.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c157d80 0x7f972c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c157d80 0x7f972c0b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40608/0 (socket says 172.21.15.117:40608) 2020-11-20T13:56:01.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 -- 172.21.15.117:0/151064130 learned_addr learned my addr 172.21.15.117:0/151064130 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:01.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9730ef3700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c066d20 0x7f972c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f97316f4700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f972c0b2cc0 0x7f972c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.864 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 -- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c066d20 msgr2=0x7f972c157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.864 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c066d20 0x7f972c157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.864 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 -- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f972c0b2cc0 msgr2=0x7f972c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:01.865 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f972c0b2cc0 0x7f972c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:01.865 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 -- 172.21.15.117:0/151064130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c0b30a0 con 0x7f972c157d80 2020-11-20T13:56:01.865 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9731ef5700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c157d80 0x7f972c0b40f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f972c155010 tx=0x7f9724008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.865 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f971a7fc700 1 -- 172.21.15.117:0/151064130 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f972400e8a0 con 0x7f972c157d80 2020-11-20T13:56:01.866 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f971a7fc700 1 -- 172.21.15.117:0/151064130 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972400ea00 con 0x7f972c157d80 2020-11-20T13:56:01.866 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f971a7fc700 1 -- 172.21.15.117:0/151064130 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f972400eda0 con 0x7f972c157d80 2020-11-20T13:56:01.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c158230 con 0x7f972c157d80 2020-11-20T13:56:01.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.834+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c0b47a0 con 0x7f972c157d80 2020-11-20T13:56:01.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.835+0000 7f971a7fc700 1 -- 172.21.15.117:0/151064130 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9724014430 con 0x7f972c157d80 2020-11-20T13:56:01.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.835+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c1575f0 con 0x7f972c157d80 2020-11-20T13:56:01.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.835+0000 7f971a7fc700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f970c051320 0x7f970c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:01.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.835+0000 7f971a7fc700 1 -- 172.21.15.117:0/151064130 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9724069920 con 0x7f972c157d80 2020-11-20T13:56:01.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.836+0000 7f97316f4700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f970c051320 0x7f970c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:01.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.836+0000 7f97316f4700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f970c051320 0x7f970c0537e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f971c017000 tx=0x7f971c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:01.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:01.839+0000 7f971a7fc700 1 -- 172.21.15.117:0/151064130 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f972c1575f0 con 0x7f972c157d80 2020-11-20T13:56:02.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.037+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f972c1575f0 con 0x7f972c157d80 2020-11-20T13:56:02.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.037+0000 7f971a7fc700 1 -- 172.21.15.117:0/151064130 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7f972c1575f0 con 0x7f972c157d80 2020-11-20T13:56:02.039 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:02.039 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:55:51.676934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/1587825839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1587825839},{"type":"v1","addr":"172.21.15.117:6835","nonce":1587825839}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:02.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.038+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f970c051320 msgr2=0x7f970c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.038+0000 7f9733958700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f970c051320 0x7f970c0537e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f971c017000 tx=0x7f971c015000).stop 2020-11-20T13:56:02.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.038+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c157d80 msgr2=0x7f972c0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.038+0000 7f9733958700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c157d80 0x7f972c0b40f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f972c155010 tx=0x7f9724008040).stop 2020-11-20T13:56:02.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 shutdown_connections 2020-11-20T13:56:02.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f970c051320 0x7f970c0537e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c157d80 0x7f972c0b40f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f972c0b2cc0 0x7f972c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 --2- 172.21.15.117:0/151064130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f972c066d20 0x7f972c157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 >> 172.21.15.117:0/151064130 conn(0x7f972c05b9f0 msgr2=0x7f972c09ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:02.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 shutdown_connections 2020-11-20T13:56:02.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.039+0000 7f9733958700 1 -- 172.21.15.117:0/151064130 wait complete. 2020-11-20T13:56:02.043 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2020-11-20T13:56:02.254 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:55:50.675265+0000', 'modified': '2020-11-20T13:55:51.676934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/1587825839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1587825839}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1587825839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:02.254 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:02.255 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:02.384 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":4288,"want_state":"up:active","state":"up:active","rank_uptime":11.698767171,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":18.088695690000002} 2020-11-20T13:56:02.385 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 4288, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 11.698767171, "state": "up:active", "uptime": 18.08869569, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:02.385 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-11-20T13:56:02.404 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-11-20T13:56:02.405 INFO:tasks.ceph_fuse:config is None 2020-11-20T13:56:02.405 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None, 'client.3': None} 2020-11-20T13:56:02.405 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-11-20T13:56:02.406 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:02.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.511+0000 7f1f80931700 1 Processor -- start 2020-11-20T13:56:02.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.511+0000 7f1f80931700 1 -- start start 2020-11-20T13:56:02.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f80931700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f1f7c098e30 0x7f1f7c09b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f80931700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f7c09b700 0x7f1f7c1438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f80931700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0b5510 con 0x7f1f7c143dd0 2020-11-20T13:56:02.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f80931700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b57b0 con 0x7f1f7c098e30 2020-11-20T13:56:02.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f80931700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b5a50 con 0x7f1f7c09b700 2020-11-20T13:56:02.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f79d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f7c09b700 0x7f1f7c1438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f7ad9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1f7c143dd0 0x7f1f7c0b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59772/0 (socket says 172.21.15.117:59772) 2020-11-20T13:56:02.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.512+0000 7f1f7ad9d700 1 -- 172.21.15.117:0/122699796 learned_addr learned my addr 172.21.15.117:0/122699796 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:02.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f79d9b700 1 -- 172.21.15.117:0/122699796 >> v2:172.21.15.85:3300/0 conn(0x7f1f7c098e30 msgr2=0x7f1f7c09b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:02.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f79d9b700 1 --2- 172.21.15.117:0/122699796 >> v2:172.21.15.85:3300/0 conn(0x7f1f7c098e30 0x7f1f7c09b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:02.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f79d9b700 1 -- 172.21.15.117:0/122699796 >> v1:172.21.15.117:6789/0 conn(0x7f1f7c143dd0 legacy=0x7f1f7c0b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f7959a700 1 -- 172.21.15.117:0/122699796 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924794187 0 0) 0x7f1f7c0b5510 con 0x7f1f7c143dd0 2020-11-20T13:56:02.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f79d9b700 1 -- 172.21.15.117:0/122699796 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c099210 con 0x7f1f7c09b700 2020-11-20T13:56:02.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f79d9b700 1 --2- 172.21.15.117:0/122699796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c09b700 0x7f1f7c1438e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f1f6c002f30 tx=0x7f1f6c006dc0).ready entity=mon.1 client_cookie=ef0f24f65a4e9a1b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:02.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f7959a700 1 -- 172.21.15.117:0/122699796 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f6c0097f0 con 0x7f1f7c09b700 2020-11-20T13:56:02.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f7959a700 1 -- 172.21.15.117:0/122699796 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c009950 con 0x7f1f7c09b700 2020-11-20T13:56:02.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f7959a700 1 -- 172.21.15.117:0/122699796 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f6c009cb0 con 0x7f1f7c09b700 2020-11-20T13:56:02.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f80931700 1 -- 172.21.15.117:0/122699796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c09b700 msgr2=0x7f1f7c1438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f80931700 1 --2- 172.21.15.117:0/122699796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c09b700 0x7f1f7c1438e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f1f6c002f30 tx=0x7f1f6c006dc0).stop 2020-11-20T13:56:02.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f80931700 1 -- 172.21.15.117:0/122699796 shutdown_connections 2020-11-20T13:56:02.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f80931700 1 --2- 172.21.15.117:0/122699796 >> v2:172.21.15.85:3300/0 conn(0x7f1f7c098e30 0x7f1f7c09b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:02.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f80931700 1 --2- 172.21.15.117:0/122699796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c09b700 0x7f1f7c1438e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f80931700 1 -- 172.21.15.117:0/122699796 >> 172.21.15.117:0/122699796 conn(0x7f1f7c05b9f0 msgr2=0x7f1f7c05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:02.525 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.513+0000 7f1f80931700 1 -- 172.21.15.117:0/122699796 shutdown_connections 2020-11-20T13:56:02.525 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.514+0000 7f1f80931700 1 -- 172.21.15.117:0/122699796 wait complete. 2020-11-20T13:56:02.527 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.514+0000 7f1f80931700 1 Processor -- start 2020-11-20T13:56:02.528 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.514+0000 7f1f80931700 1 -- start start 2020-11-20T13:56:02.528 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f80931700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f7c098e30 0x7f1f7c0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.528 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f80931700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f7c09b700 0x7f1f7c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.528 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f80931700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c143dd0 0x7f1f7c09e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.529 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f80931700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c09db00 con 0x7f1f7c09b700 2020-11-20T13:56:02.529 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f80931700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c09c7a0 con 0x7f1f7c143dd0 2020-11-20T13:56:02.529 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f80931700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c09c9d0 con 0x7f1f7c098e30 2020-11-20T13:56:02.529 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c143dd0 0x7f1f7c09e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.529 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c143dd0 0x7f1f7c09e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40620/0 (socket says 172.21.15.117:40620) 2020-11-20T13:56:02.530 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 -- 172.21.15.117:0/2468217441 learned_addr learned my addr 172.21.15.117:0/2468217441 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:02.530 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7a59c700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f7c098e30 0x7f1f7c0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.530 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f79d9b700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f7c09b700 0x7f1f7c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.530 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 -- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f7c098e30 msgr2=0x7f1f7c0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.530 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f7c098e30 0x7f1f7c0b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.531 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 -- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f7c09b700 msgr2=0x7f1f7c157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.531 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f7c09b700 0x7f1f7c157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.531 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.515+0000 7f1f7ad9d700 1 -- 172.21.15.117:0/2468217441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c09cc70 con 0x7f1f7c143dd0 2020-11-20T13:56:02.531 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f7a59c700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f7c098e30 0x7f1f7c0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:02.531 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f79d9b700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f7c09b700 0x7f1f7c157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:02.532 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f7ad9d700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c143dd0 0x7f1f7c09e1e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f1f7c09e0b0 tx=0x7f1f74007cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:02.532 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f6b7fe700 1 -- 172.21.15.117:0/2468217441 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f74005b70 con 0x7f1f7c143dd0 2020-11-20T13:56:02.532 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f6b7fe700 1 -- 172.21.15.117:0/2468217441 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f74005cd0 con 0x7f1f7c143dd0 2020-11-20T13:56:02.532 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c09cf10 con 0x7f1f7c143dd0 2020-11-20T13:56:02.532 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f6b7fe700 1 -- 172.21.15.117:0/2468217441 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f74007300 con 0x7f1f7c143dd0 2020-11-20T13:56:02.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.516+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c0630a0 con 0x7f1f7c143dd0 2020-11-20T13:56:02.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.517+0000 7f1f6b7fe700 1 -- 172.21.15.117:0/2468217441 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1f74010030 con 0x7f1f7c143dd0 2020-11-20T13:56:02.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.517+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c057050 con 0x7f1f7c143dd0 2020-11-20T13:56:02.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.517+0000 7f1f6b7fe700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f58051320 0x7f1f580537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.517+0000 7f1f6b7fe700 1 -- 172.21.15.117:0/2468217441 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f1f740691d0 con 0x7f1f7c143dd0 2020-11-20T13:56:02.534 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.517+0000 7f1f7a59c700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f58051320 0x7f1f580537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.534 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.517+0000 7f1f7a59c700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f58051320 0x7f1f580537e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f1f7000a000 tx=0x7f1f70008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:02.535 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.520+0000 7f1f6b7fe700 1 -- 172.21.15.117:0/2468217441 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1f74066a60 con 0x7f1f7c143dd0 2020-11-20T13:56:02.672 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.671+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f7c09c060 con 0x7f1f7c143dd0 2020-11-20T13:56:02.673 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.672+0000 7f1f6b7fe700 1 -- 172.21.15.117:0/2468217441 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f1f74041280 con 0x7f1f7c143dd0 2020-11-20T13:56:02.673 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:02.674 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:02.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f58051320 msgr2=0x7f1f580537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f58051320 0x7f1f580537e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f1f7000a000 tx=0x7f1f70008040).stop 2020-11-20T13:56:02.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c143dd0 msgr2=0x7f1f7c09e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c143dd0 0x7f1f7c09e1e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f1f7c09e0b0 tx=0x7f1f74007cb0).stop 2020-11-20T13:56:02.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 shutdown_connections 2020-11-20T13:56:02.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f58051320 0x7f1f580537e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f7c098e30 0x7f1f7c0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f7c143dd0 0x7f1f7c09e1e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 --2- 172.21.15.117:0/2468217441 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f7c09b700 0x7f1f7c157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 >> 172.21.15.117:0/2468217441 conn(0x7f1f7c05b9f0 msgr2=0x7f1f7c0b8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:02.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 shutdown_connections 2020-11-20T13:56:02.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.674+0000 7f1f80931700 1 -- 172.21.15.117:0/2468217441 wait complete. 2020-11-20T13:56:02.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:02.687 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:02.795 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.794+0000 7f82e2b70700 1 Processor -- start 2020-11-20T13:56:02.796 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.794+0000 7f82e2b70700 1 -- start start 2020-11-20T13:56:02.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e2b70700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc09b930 0x7f82dc149420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.799 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e2b70700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f82dc098dc0 0x7f82dc145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.799 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e2b70700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc148a40 con 0x7f82dc1447b0 2020-11-20T13:56:02.799 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e2b70700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc1462f0 con 0x7f82dc098dc0 2020-11-20T13:56:02.799 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e2b70700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc148ce0 con 0x7f82dc09b930 2020-11-20T13:56:02.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82dbfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f82dc098dc0 0x7f82dc145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e110d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f82dc1447b0 0x7f82dc144c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59788/0 (socket says 172.21.15.117:59788) 2020-11-20T13:56:02.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e110d700 1 -- 172.21.15.117:0/91574122 learned_addr learned my addr 172.21.15.117:0/91574122 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:02.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e090c700 1 --2- 172.21.15.117:0/91574122 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc09b930 0x7f82dc149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82dbfff700 1 -- 172.21.15.117:0/91574122 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc09b930 msgr2=0x7f82dc149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:02.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82dbfff700 1 --2- 172.21.15.117:0/91574122 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc09b930 0x7f82dc149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82dbfff700 1 -- 172.21.15.117:0/91574122 >> v1:172.21.15.117:6789/0 conn(0x7f82dc1447b0 legacy=0x7f82dc144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82dbfff700 1 -- 172.21.15.117:0/91574122 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc148f80 con 0x7f82dc098dc0 2020-11-20T13:56:02.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82db7fe700 1 -- 172.21.15.117:0/91574122 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88270614 0 0) 0x7f82dc148a40 con 0x7f82dc1447b0 2020-11-20T13:56:02.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82e090c700 1 --2- 172.21.15.117:0/91574122 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc09b930 0x7f82dc149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:02.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82dbfff700 1 --2- 172.21.15.117:0/91574122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc098dc0 0x7f82dc145e00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f82cc002f30 tx=0x7f82cc006dc0).ready entity=mon.1 client_cookie=bacaa16c2f2ffba8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:02.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.795+0000 7f82db7fe700 1 -- 172.21.15.117:0/91574122 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82cc0097f0 con 0x7f82dc098dc0 2020-11-20T13:56:02.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82db7fe700 1 -- 172.21.15.117:0/91574122 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc009950 con 0x7f82dc098dc0 2020-11-20T13:56:02.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82db7fe700 1 -- 172.21.15.117:0/91574122 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82cc009cb0 con 0x7f82dc098dc0 2020-11-20T13:56:02.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 -- 172.21.15.117:0/91574122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc098dc0 msgr2=0x7f82dc145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 --2- 172.21.15.117:0/91574122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc098dc0 0x7f82dc145e00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f82cc002f30 tx=0x7f82cc006dc0).stop 2020-11-20T13:56:02.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 -- 172.21.15.117:0/91574122 shutdown_connections 2020-11-20T13:56:02.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 --2- 172.21.15.117:0/91574122 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc09b930 0x7f82dc149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 --2- 172.21.15.117:0/91574122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc098dc0 0x7f82dc145e00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 -- 172.21.15.117:0/91574122 >> 172.21.15.117:0/91574122 conn(0x7f82dc0b9770 msgr2=0x7f82dc0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:02.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 -- 172.21.15.117:0/91574122 shutdown_connections 2020-11-20T13:56:02.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.796+0000 7f82e2b70700 1 -- 172.21.15.117:0/91574122 wait complete. 2020-11-20T13:56:02.805 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.797+0000 7f82e2b70700 1 Processor -- start 2020-11-20T13:56:02.805 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.797+0000 7f82e2b70700 1 -- start start 2020-11-20T13:56:02.807 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e2b70700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc098dc0 0x7f82dc0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e2b70700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82dc09b930 0x7f82dc0677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e2b70700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc1447b0 0x7f82dc1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e2b70700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b12a0 con 0x7f82dc09b930 2020-11-20T13:56:02.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e2b70700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b1540 con 0x7f82dc1447b0 2020-11-20T13:56:02.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e2b70700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc063bf0 con 0x7f82dc098dc0 2020-11-20T13:56:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc1447b0 0x7f82dc1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc1447b0 0x7f82dc1578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40634/0 (socket says 172.21.15.117:40634) 2020-11-20T13:56:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 -- 172.21.15.117:0/435793064 learned_addr learned my addr 172.21.15.117:0/435793064 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:02.809 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e090c700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc098dc0 0x7f82dc0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.810 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82dbfff700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82dc09b930 0x7f82dc0677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.810 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 -- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc098dc0 msgr2=0x7f82dc0aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.810 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc098dc0 0x7f82dc0aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.810 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 -- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82dc09b930 msgr2=0x7f82dc0677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.810 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82dc09b930 0x7f82dc0677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 -- 172.21.15.117:0/435793064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc158460 con 0x7f82dc1447b0 2020-11-20T13:56:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82e110d700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc1447b0 0x7f82dc1578e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f82c4000b80 tx=0x7f82c4007930).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.798+0000 7f82d9ffb700 1 -- 172.21.15.117:0/435793064 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82c40072e0 con 0x7f82dc1447b0 2020-11-20T13:56:02.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.799+0000 7f82d9ffb700 1 -- 172.21.15.117:0/435793064 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4007440 con 0x7f82dc1447b0 2020-11-20T13:56:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.799+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc0b0e70 con 0x7f82dc1447b0 2020-11-20T13:56:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.799+0000 7f82d9ffb700 1 -- 172.21.15.117:0/435793064 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82c40057d0 con 0x7f82dc1447b0 2020-11-20T13:56:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.799+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc157f90 con 0x7f82dc1447b0 2020-11-20T13:56:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.799+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc063db0 con 0x7f82dc1447b0 2020-11-20T13:56:02.812 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.800+0000 7f82d9ffb700 1 -- 172.21.15.117:0/435793064 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f82c4010070 con 0x7f82dc1447b0 2020-11-20T13:56:02.813 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.800+0000 7f82d9ffb700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f82c00513c0 0x7f82c0053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:02.813 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.800+0000 7f82d9ffb700 1 -- 172.21.15.117:0/435793064 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f82c40687e0 con 0x7f82dc1447b0 2020-11-20T13:56:02.813 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.800+0000 7f82e090c700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f82c00513c0 0x7f82c0053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:02.813 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.800+0000 7f82e090c700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f82c00513c0 0x7f82c0053880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f82d400b000 tx=0x7f82d4008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:02.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.803+0000 7f82d9ffb700 1 -- 172.21.15.117:0/435793064 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f82c4040a00 con 0x7f82dc1447b0 2020-11-20T13:56:02.956 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.954+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f82dc057050 con 0x7f82dc1447b0 2020-11-20T13:56:02.956 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.954+0000 7f82d9ffb700 1 -- 172.21.15.117:0/435793064 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f82c4044020 con 0x7f82dc1447b0 2020-11-20T13:56:02.957 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:02.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.955+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f82c00513c0 msgr2=0x7f82c0053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.955+0000 7f82e2b70700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f82c00513c0 0x7f82c0053880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f82d400b000 tx=0x7f82d4008040).stop 2020-11-20T13:56:02.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.955+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc1447b0 msgr2=0x7f82dc1578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:02.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.955+0000 7f82e2b70700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc1447b0 0x7f82dc1578e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f82c4000b80 tx=0x7f82c4007930).stop 2020-11-20T13:56:02.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 shutdown_connections 2020-11-20T13:56:02.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f82c00513c0 0x7f82c0053880 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82dc098dc0 0x7f82dc0aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82dc1447b0 0x7f82dc1578e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 --2- 172.21.15.117:0/435793064 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82dc09b930 0x7f82dc0677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:02.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 >> 172.21.15.117:0/435793064 conn(0x7f82dc0b9770 msgr2=0x7f82dc0b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:02.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 shutdown_connections 2020-11-20T13:56:02.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:02.956+0000 7f82e2b70700 1 -- 172.21.15.117:0/435793064 wait complete. 2020-11-20T13:56:02.967 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:03.075 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.073+0000 7fd8e4d19700 1 Processor -- start 2020-11-20T13:56:03.075 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.073+0000 7fd8e4d19700 1 -- start start 2020-11-20T13:56:03.077 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.074+0000 7fd8e4d19700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e0098d60 0x7fd8e009b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.074+0000 7fd8e4d19700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd8e00b56a0 0x7fd8e009c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.074+0000 7fd8e4d19700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00b4860 con 0x7fd8e00b5240 2020-11-20T13:56:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.074+0000 7fd8e4d19700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e01439f0 con 0x7fd8e00b56a0 2020-11-20T13:56:03.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.074+0000 7fd8e4d19700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00b4b00 con 0x7fd8e0098d60 2020-11-20T13:56:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8de59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e0098d60 0x7fd8e009b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8ded9d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd8e00b56a0 0x7fd8e009c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8de59c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e0098d60 0x7fd8e009b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:40704/0 (socket says 172.21.15.117:40704) 2020-11-20T13:56:03.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8de59c700 1 -- 172.21.15.117:0/2217025539 learned_addr learned my addr 172.21.15.117:0/2217025539 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8ded9d700 1 -- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e0098d60 msgr2=0x7fd8e009b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8ded9d700 1 --2- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e0098d60 0x7fd8e009b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8ded9d700 1 -- 172.21.15.117:0/2217025539 >> v1:172.21.15.85:6789/0 conn(0x7fd8e00b5240 legacy=0x7fd8e00630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8ded9d700 1 -- 172.21.15.117:0/2217025539 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00b4da0 con 0x7fd8e00b56a0 2020-11-20T13:56:03.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8de59c700 1 --2- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e0098d60 0x7fd8e009b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:03.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8ded9d700 1 --2- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 0x7fd8e009c740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd8d80032b0 tx=0x7fd8d80091a0).ready entity=mon.0 client_cookie=a36fce5574ea1a0d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8dd59a700 1 -- 172.21.15.117:0/2217025539 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d800b9a0 con 0x7fd8e00b56a0 2020-11-20T13:56:03.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8dd59a700 1 -- 172.21.15.117:0/2217025539 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d800bb00 con 0x7fd8e00b56a0 2020-11-20T13:56:03.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8dd59a700 1 -- 172.21.15.117:0/2217025539 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d800bc70 con 0x7fd8e00b56a0 2020-11-20T13:56:03.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 msgr2=0x7fd8e009c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.075+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 0x7fd8e009c740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd8d80032b0 tx=0x7fd8d80091a0).stop 2020-11-20T13:56:03.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.076+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2217025539 shutdown_connections 2020-11-20T13:56:03.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.076+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 0x7fd8e009c740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.076+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2217025539 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e0098d60 0x7fd8e009b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.076+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2217025539 >> 172.21.15.117:0/2217025539 conn(0x7fd8e00b9770 msgr2=0x7fd8e00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:03.083 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.076+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2217025539 shutdown_connections 2020-11-20T13:56:03.083 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.076+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2217025539 wait complete. 2020-11-20T13:56:03.086 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.077+0000 7fd8e4d19700 1 Processor -- start 2020-11-20T13:56:03.086 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.077+0000 7fd8e4d19700 1 -- start start 2020-11-20T13:56:03.086 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.077+0000 7fd8e4d19700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e0098d60 0x7fd8e005f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.087 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8e4d19700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e00b5240 0x7fd8e00664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.087 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8e4d19700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 0x7fd8e0067110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.087 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8e4d19700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e01585e0 con 0x7fd8e00b56a0 2020-11-20T13:56:03.087 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8e4d19700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00b06f0 con 0x7fd8e0098d60 2020-11-20T13:56:03.087 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8e4d19700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00b0990 con 0x7fd8e00b5240 2020-11-20T13:56:03.088 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e0098d60 0x7fd8e005f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.088 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e0098d60 0x7fd8e005f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40644/0 (socket says 172.21.15.117:40644) 2020-11-20T13:56:03.088 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 -- 172.21.15.117:0/2479611284 learned_addr learned my addr 172.21.15.117:0/2479611284 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:03.088 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8ddd9b700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e00b5240 0x7fd8e00664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.089 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8ded9d700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 0x7fd8e0067110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.089 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 -- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e00b5240 msgr2=0x7fd8e00664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.089 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e00b5240 0x7fd8e00664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.089 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 -- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 msgr2=0x7fd8e0067110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.089 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 0x7fd8e0067110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.090 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 -- 172.21.15.117:0/2479611284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e0158230 con 0x7fd8e0098d60 2020-11-20T13:56:03.090 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8de59c700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e0098d60 0x7fd8e005f2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fd8d400c000 tx=0x7fd8d4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.090 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8cf7fe700 1 -- 172.21.15.117:0/2479611284 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d400eea0 con 0x7fd8e0098d60 2020-11-20T13:56:03.090 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8cf7fe700 1 -- 172.21.15.117:0/2479611284 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d400cbb0 con 0x7fd8e0098d60 2020-11-20T13:56:03.090 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e00af0f0 con 0x7fd8e0098d60 2020-11-20T13:56:03.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8cf7fe700 1 -- 172.21.15.117:0/2479611284 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8d40123f0 con 0x7fd8e0098d60 2020-11-20T13:56:03.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.078+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e0066ba0 con 0x7fd8e0098d60 2020-11-20T13:56:03.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.079+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e0158750 con 0x7fd8e0098d60 2020-11-20T13:56:03.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.079+0000 7fd8cf7fe700 1 -- 172.21.15.117:0/2479611284 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd8d4022070 con 0x7fd8e0098d60 2020-11-20T13:56:03.091 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.080+0000 7fd8cf7fe700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd8bc0513c0 0x7fd8bc053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.080+0000 7fd8cf7fe700 1 -- 172.21.15.117:0/2479611284 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd8d4069810 con 0x7fd8e0098d60 2020-11-20T13:56:03.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.080+0000 7fd8ddd9b700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd8bc0513c0 0x7fd8bc053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.080+0000 7fd8ddd9b700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd8bc0513c0 0x7fd8bc053880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd8d000b000 tx=0x7fd8d0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.092 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.083+0000 7fd8cf7fe700 1 -- 172.21.15.117:0/2479611284 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd8e0158750 con 0x7fd8e0098d60 2020-11-20T13:56:03.234 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.233+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd8e0158750 con 0x7fd8e0098d60 2020-11-20T13:56:03.234 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.233+0000 7fd8cf7fe700 1 -- 172.21.15.117:0/2479611284 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd8e0158750 con 0x7fd8e0098d60 2020-11-20T13:56:03.235 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:03.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd8bc0513c0 msgr2=0x7fd8bc053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd8bc0513c0 0x7fd8bc053880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fd8d000b000 tx=0x7fd8d0009040).stop 2020-11-20T13:56:03.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e0098d60 msgr2=0x7fd8e005f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e0098d60 0x7fd8e005f2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fd8d400c000 tx=0x7fd8d4008040).stop 2020-11-20T13:56:03.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 shutdown_connections 2020-11-20T13:56:03.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd8bc0513c0 0x7fd8bc053880 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd8e00b56a0 0x7fd8e0067110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8e0098d60 0x7fd8e005f2e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 --2- 172.21.15.117:0/2479611284 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd8e00b5240 0x7fd8e00664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 >> 172.21.15.117:0/2479611284 conn(0x7fd8e00b9770 msgr2=0x7fd8e013d140 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:03.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 shutdown_connections 2020-11-20T13:56:03.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.235+0000 7fd8e4d19700 1 -- 172.21.15.117:0/2479611284 wait complete. 2020-11-20T13:56:03.246 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:03.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.353+0000 7f4361d79700 1 Processor -- start 2020-11-20T13:56:03.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.353+0000 7f4361d79700 1 -- start start 2020-11-20T13:56:03.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f4361d79700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f435c09bd80 0x7f435c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f4361d79700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c0999d0 con 0x7f435c143eb0 2020-11-20T13:56:03.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f4361d79700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c099c70 con 0x7f435c09a110 2020-11-20T13:56:03.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f4361d79700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c09c160 con 0x7f435c09bd80 2020-11-20T13:56:03.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f435b7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f435c09bd80 0x7f435c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f435affd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f435c09a110 0x7f435c1437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59812/0 (socket says 172.21.15.117:59812) 2020-11-20T13:56:03.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f435affd700 1 -- 172.21.15.117:0/3786957291 learned_addr learned my addr 172.21.15.117:0/3786957291 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:03.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f435b7fe700 1 -- 172.21.15.117:0/3786957291 >> v1:172.21.15.117:6789/0 conn(0x7f435c09a110 legacy=0x7f435c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f435b7fe700 1 -- 172.21.15.117:0/3786957291 >> v1:172.21.15.85:6789/0 conn(0x7f435c143eb0 legacy=0x7f435c0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.354+0000 7f435b7fe700 1 -- 172.21.15.117:0/3786957291 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c09a5c0 con 0x7f435c09bd80 2020-11-20T13:56:03.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.355+0000 7f435b7fe700 1 --2- 172.21.15.117:0/3786957291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09bd80 0x7f435c09cb40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f434c002f30 tx=0x7f434c006dc0).ready entity=mon.1 client_cookie=70ce42cf27d0f8ca server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.355+0000 7f435a7fc700 1 -- 172.21.15.117:0/3786957291 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f434c0097f0 con 0x7f435c09bd80 2020-11-20T13:56:03.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.355+0000 7f435a7fc700 1 -- 172.21.15.117:0/3786957291 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c009950 con 0x7f435c09bd80 2020-11-20T13:56:03.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.355+0000 7f435a7fc700 1 -- 172.21.15.117:0/3786957291 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f434c009cb0 con 0x7f435c09bd80 2020-11-20T13:56:03.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.355+0000 7f4361d79700 1 -- 172.21.15.117:0/3786957291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09bd80 msgr2=0x7f435c09cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.355+0000 7f4361d79700 1 --2- 172.21.15.117:0/3786957291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09bd80 0x7f435c09cb40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f434c002f30 tx=0x7f434c006dc0).stop 2020-11-20T13:56:03.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.356+0000 7f4361d79700 1 -- 172.21.15.117:0/3786957291 shutdown_connections 2020-11-20T13:56:03.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.356+0000 7f4361d79700 1 --2- 172.21.15.117:0/3786957291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09bd80 0x7f435c09cb40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.356+0000 7f4361d79700 1 -- 172.21.15.117:0/3786957291 >> 172.21.15.117:0/3786957291 conn(0x7f435c0b9770 msgr2=0x7f435c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:03.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.356+0000 7f4361d79700 1 -- 172.21.15.117:0/3786957291 shutdown_connections 2020-11-20T13:56:03.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.356+0000 7f4361d79700 1 -- 172.21.15.117:0/3786957291 wait complete. 2020-11-20T13:56:03.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 Processor -- start 2020-11-20T13:56:03.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 -- start start 2020-11-20T13:56:03.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09a110 0x7f435c067070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f435c09bd80 0x7f435c0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f435c143eb0 0x7f435c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0b1740 con 0x7f435c09bd80 2020-11-20T13:56:03.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1579f0 con 0x7f435c09a110 2020-11-20T13:56:03.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f4361d79700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c157c90 con 0x7f435c143eb0 2020-11-20T13:56:03.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f435b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09a110 0x7f435c067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.357+0000 7f435b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09a110 0x7f435c067070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40658/0 (socket says 172.21.15.117:40658) 2020-11-20T13:56:03.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435b7fe700 1 -- 172.21.15.117:0/2146169662 learned_addr learned my addr 172.21.15.117:0/2146169662 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:03.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435affd700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f435c09bd80 0x7f435c0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435bfff700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f435c143eb0 0x7f435c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435b7fe700 1 -- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f435c143eb0 msgr2=0x7f435c158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435b7fe700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f435c143eb0 0x7f435c158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435b7fe700 1 -- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f435c09bd80 msgr2=0x7f435c0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435b7fe700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f435c09bd80 0x7f435c0b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435b7fe700 1 -- 172.21.15.117:0/2146169662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c1444c0 con 0x7f435c09a110 2020-11-20T13:56:03.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f435b7fe700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09a110 0x7f435c067070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f434c002f30 tx=0x7f434c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f4358ff9700 1 -- 172.21.15.117:0/2146169662 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f434c011e30 con 0x7f435c09a110 2020-11-20T13:56:03.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f4358ff9700 1 -- 172.21.15.117:0/2146169662 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c01d450 con 0x7f435c09a110 2020-11-20T13:56:03.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c0b12d0 con 0x7f435c09a110 2020-11-20T13:56:03.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f4358ff9700 1 -- 172.21.15.117:0/2146169662 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f434c01d7b0 con 0x7f435c09a110 2020-11-20T13:56:03.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.358+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c1587e0 con 0x7f435c09a110 2020-11-20T13:56:03.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.359+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c057050 con 0x7f435c09a110 2020-11-20T13:56:03.370 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.359+0000 7f4358ff9700 1 -- 172.21.15.117:0/2146169662 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f434c024030 con 0x7f435c09a110 2020-11-20T13:56:03.370 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.359+0000 7f4358ff9700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f433c051320 0x7f433c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.359+0000 7f4358ff9700 1 -- 172.21.15.117:0/2146169662 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f434c06a3e0 con 0x7f435c09a110 2020-11-20T13:56:03.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.359+0000 7f435affd700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f433c051320 0x7f433c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.360+0000 7f435affd700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f433c051320 0x7f433c0537e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f435000d000 tx=0x7f4350009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.372 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.363+0000 7f4358ff9700 1 -- 172.21.15.117:0/2146169662 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f434c042640 con 0x7f435c09a110 2020-11-20T13:56:03.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.513+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f435c157e50 con 0x7f435c09a110 2020-11-20T13:56:03.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.513+0000 7f4358ff9700 1 -- 172.21.15.117:0/2146169662 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f434c045c20 con 0x7f435c09a110 2020-11-20T13:56:03.515 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:03.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f433c051320 msgr2=0x7f433c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f433c051320 0x7f433c0537e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f435000d000 tx=0x7f4350009040).stop 2020-11-20T13:56:03.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09a110 msgr2=0x7f435c067070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09a110 0x7f435c067070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f434c002f30 tx=0x7f434c015000).stop 2020-11-20T13:56:03.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 shutdown_connections 2020-11-20T13:56:03.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f433c051320 0x7f433c0537e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f435c143eb0 0x7f435c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f435c09a110 0x7f435c067070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 --2- 172.21.15.117:0/2146169662 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f435c09bd80 0x7f435c0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 >> 172.21.15.117:0/2146169662 conn(0x7f435c0b9770 msgr2=0x7f435c0b9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:03.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 shutdown_connections 2020-11-20T13:56:03.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.515+0000 7f4361d79700 1 -- 172.21.15.117:0/2146169662 wait complete. 2020-11-20T13:56:03.526 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:03.636 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.634+0000 7f91b6587700 1 Processor -- start 2020-11-20T13:56:03.636 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.634+0000 7f91b6587700 1 -- start start 2020-11-20T13:56:03.638 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91b6587700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b009cb40 0x7f91b01437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.638 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91b6587700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00999d0 con 0x7f91b009bd80 2020-11-20T13:56:03.638 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91b6587700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b0099c70 con 0x7f91b0143c90 2020-11-20T13:56:03.639 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91b6587700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00997f0 con 0x7f91b009cb40 2020-11-20T13:56:03.639 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91b4b24700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f91b0143c90 0x7f91b00b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59830/0 (socket says 172.21.15.117:59830) 2020-11-20T13:56:03.639 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91b4b24700 1 -- 172.21.15.117:0/540253594 learned_addr learned my addr 172.21.15.117:0/540253594 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:03.639 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91af7fe700 1 --2- 172.21.15.117:0/540253594 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b009cb40 0x7f91b01437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.639 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856936113 0 0) 0x7f91b0099c70 con 0x7f91b0143c90 2020-11-20T13:56:03.640 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f919c003230 con 0x7f91b0143c90 2020-11-20T13:56:03.640 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827114681 0 0) 0x7f91b00999d0 con 0x7f91b009bd80 2020-11-20T13:56:03.640 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.635+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b0099c70 con 0x7f91b009bd80 2020-11-20T13:56:03.640 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91af7fe700 1 -- 172.21.15.117:0/540253594 >> v1:172.21.15.117:6789/0 conn(0x7f91b0143c90 legacy=0x7f91b00b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.640 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 103021031 0 0) 0x7f919c003230 con 0x7f91b0143c90 2020-11-20T13:56:03.641 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91af7fe700 1 -- 172.21.15.117:0/540253594 >> v1:172.21.15.85:6789/0 conn(0x7f91b009bd80 legacy=0x7f91b009a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.641 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91af7fe700 1 -- 172.21.15.117:0/540253594 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0144120 con 0x7f91b009cb40 2020-11-20T13:56:03.641 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91af7fe700 1 --2- 172.21.15.117:0/540253594 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b009cb40 0x7f91b01437a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f91b009b9c0 tx=0x7f91a4006dc0).ready entity=mon.2 client_cookie=36ab82f29196acf9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.641 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91a8001a50 con 0x7f91b009cb40 2020-11-20T13:56:03.641 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a400d5c0 con 0x7f91b009cb40 2020-11-20T13:56:03.642 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91aeffd700 1 -- 172.21.15.117:0/540253594 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91a400d730 con 0x7f91b009cb40 2020-11-20T13:56:03.642 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91b6587700 1 -- 172.21.15.117:0/540253594 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b009cb40 msgr2=0x7f91b01437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.642 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91b6587700 1 --2- 172.21.15.117:0/540253594 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b009cb40 0x7f91b01437a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f91b009b9c0 tx=0x7f91a4006dc0).stop 2020-11-20T13:56:03.642 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91b6587700 1 -- 172.21.15.117:0/540253594 shutdown_connections 2020-11-20T13:56:03.642 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91b6587700 1 --2- 172.21.15.117:0/540253594 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b009cb40 0x7f91b01437a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.643 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91b6587700 1 -- 172.21.15.117:0/540253594 >> 172.21.15.117:0/540253594 conn(0x7f91b00b9770 msgr2=0x7f91b00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:03.643 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91b6587700 1 -- 172.21.15.117:0/540253594 shutdown_connections 2020-11-20T13:56:03.643 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.636+0000 7f91b6587700 1 -- 172.21.15.117:0/540253594 wait complete. 2020-11-20T13:56:03.647 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.637+0000 7f91b6587700 1 Processor -- start 2020-11-20T13:56:03.647 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.637+0000 7f91b6587700 1 -- start start 2020-11-20T13:56:03.647 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91b6587700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b009bd80 0x7f91b005f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.647 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91b6587700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91b009cb40 0x7f91b013f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.648 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91b6587700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b0143c90 0x7f91b00b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.648 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91b6587700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00b1890 con 0x7f91b009cb40 2020-11-20T13:56:03.648 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91b6587700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b01588c0 con 0x7f91b009bd80 2020-11-20T13:56:03.648 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91b6587700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00b1ac0 con 0x7f91b0143c90 2020-11-20T13:56:03.648 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b009bd80 0x7f91b005f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.649 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b009bd80 0x7f91b005f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40672/0 (socket says 172.21.15.117:40672) 2020-11-20T13:56:03.649 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.638+0000 7f91affff700 1 -- 172.21.15.117:0/3017947781 learned_addr learned my addr 172.21.15.117:0/3017947781 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:03.649 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91af7fe700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91b009cb40 0x7f91b013f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.649 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91b4b24700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b0143c90 0x7f91b00b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.649 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91affff700 1 -- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b0143c90 msgr2=0x7f91b00b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.650 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91affff700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b0143c90 0x7f91b00b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.650 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91affff700 1 -- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91b009cb40 msgr2=0x7f91b013f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.650 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91affff700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91b009cb40 0x7f91b013f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.650 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91affff700 1 -- 172.21.15.117:0/3017947781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b005f6d0 con 0x7f91b009bd80 2020-11-20T13:56:03.650 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91affff700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b009bd80 0x7f91b005f1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f91a00010d0 tx=0x7f91a0007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.651 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91ad7fa700 1 -- 172.21.15.117:0/3017947781 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91a000f470 con 0x7f91b009bd80 2020-11-20T13:56:03.651 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91ad7fa700 1 -- 172.21.15.117:0/3017947781 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a000f5d0 con 0x7f91b009bd80 2020-11-20T13:56:03.651 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b00b2e00 con 0x7f91b009bd80 2020-11-20T13:56:03.652 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91ad7fa700 1 -- 172.21.15.117:0/3017947781 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91a000f930 con 0x7f91b009bd80 2020-11-20T13:56:03.652 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.639+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b0158690 con 0x7f91b009bd80 2020-11-20T13:56:03.652 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.640+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b0057050 con 0x7f91b009bd80 2020-11-20T13:56:03.652 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.640+0000 7f91ad7fa700 1 -- 172.21.15.117:0/3017947781 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f91a0019030 con 0x7f91b009bd80 2020-11-20T13:56:03.652 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.640+0000 7f91ad7fa700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f91900513c0 0x7f9190053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.653 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.641+0000 7f91ad7fa700 1 -- 172.21.15.117:0/3017947781 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f91a00684d0 con 0x7f91b009bd80 2020-11-20T13:56:03.653 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.641+0000 7f91af7fe700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f91900513c0 0x7f9190053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.653 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.641+0000 7f91af7fe700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f91900513c0 0x7f9190053880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f91a401c000 tx=0x7f91a4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.653 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.644+0000 7f91ad7fa700 1 -- 172.21.15.117:0/3017947781 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f91a00406f0 con 0x7f91b009bd80 2020-11-20T13:56:03.795 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.793+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91b0099510 con 0x7f91b009bd80 2020-11-20T13:56:03.795 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.794+0000 7f91ad7fa700 1 -- 172.21.15.117:0/3017947781 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f91a0043d10 con 0x7f91b009bd80 2020-11-20T13:56:03.795 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:03.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f91900513c0 msgr2=0x7f9190053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f91900513c0 0x7f9190053880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f91a401c000 tx=0x7f91a4015000).stop 2020-11-20T13:56:03.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b009bd80 msgr2=0x7f91b005f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b009bd80 0x7f91b005f1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f91a00010d0 tx=0x7f91a0007a20).stop 2020-11-20T13:56:03.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 shutdown_connections 2020-11-20T13:56:03.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f91900513c0 0x7f9190053880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91b0143c90 0x7f91b00b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b009bd80 0x7f91b005f1e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 --2- 172.21.15.117:0/3017947781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91b009cb40 0x7f91b013f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.799 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 >> 172.21.15.117:0/3017947781 conn(0x7f91b00b9770 msgr2=0x7f91b01425f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:03.799 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 shutdown_connections 2020-11-20T13:56:03.799 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.795+0000 7f91b6587700 1 -- 172.21.15.117:0/3017947781 wait complete. 2020-11-20T13:56:03.807 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:03.807 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:03.914 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.911+0000 7f55ed457700 1 Processor -- start 2020-11-20T13:56:03.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.911+0000 7f55ed457700 1 -- start start 2020-11-20T13:56:03.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.911+0000 7f55ed457700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e80aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.911+0000 7f55ed457700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f55e8099fa0 0x7f55e809a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.911+0000 7f55ed457700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e80b5120 con 0x7f55e80b5860 2020-11-20T13:56:03.915 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.911+0000 7f55ed457700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e80b4f10 con 0x7f55e8099fa0 2020-11-20T13:56:03.916 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.911+0000 7f55ed457700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80b53c0 con 0x7f55e80ae8b0 2020-11-20T13:56:03.916 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e77fe700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f55e8099fa0 0x7f55e809a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.916 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e80aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.916 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e77fe700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f55e8099fa0 0x7f55e809a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:52310/0 (socket says 172.21.15.117:52310) 2020-11-20T13:56:03.916 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e80aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:40746/0 (socket says 172.21.15.117:40746) 2020-11-20T13:56:03.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e77fe700 1 -- 172.21.15.117:0/3791437478 learned_addr learned my addr 172.21.15.117:0/3791437478 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:03.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e67fc700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f55e80b5860 0x7f55e8098b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:43744/0 (socket says 172.21.15.117:43744) 2020-11-20T13:56:03.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e5ffb700 1 -- 172.21.15.117:0/3791437478 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019627732 0 0) 0x7f55e80b5120 con 0x7f55e80b5860 2020-11-20T13:56:03.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e5ffb700 1 -- 172.21.15.117:0/3791437478 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc003230 con 0x7f55e80b5860 2020-11-20T13:56:03.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e6ffd700 1 -- 172.21.15.117:0/3791437478 >> v2:172.21.15.85:3300/0 conn(0x7f55e8099fa0 msgr2=0x7f55e809a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:03.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e6ffd700 1 --2- 172.21.15.117:0/3791437478 >> v2:172.21.15.85:3300/0 conn(0x7f55e8099fa0 0x7f55e809a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e6ffd700 1 -- 172.21.15.117:0/3791437478 >> v1:172.21.15.85:6789/0 conn(0x7f55e80b5860 legacy=0x7f55e8098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.919 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e6ffd700 1 -- 172.21.15.117:0/3791437478 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80b4ac0 con 0x7f55e80ae8b0 2020-11-20T13:56:03.919 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e77fe700 1 --2- 172.21.15.117:0/3791437478 >> v2:172.21.15.85:3300/0 conn(0x7f55e8099fa0 0x7f55e809a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:03.919 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e6ffd700 1 --2- 172.21.15.117:0/3791437478 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e80aec90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f55dc002f30 tx=0x7f55dc006dc0).ready entity=mon.2 client_cookie=614d8c844f53d635 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.920 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e5ffb700 1 -- 172.21.15.117:0/3791437478 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc0096a0 con 0x7f55e80ae8b0 2020-11-20T13:56:03.920 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e5ffb700 1 -- 172.21.15.117:0/3791437478 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc009800 con 0x7f55e80ae8b0 2020-11-20T13:56:03.920 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55e5ffb700 1 -- 172.21.15.117:0/3791437478 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc009b60 con 0x7f55e80ae8b0 2020-11-20T13:56:03.920 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55ed457700 1 -- 172.21.15.117:0/3791437478 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 msgr2=0x7f55e80aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.920 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.912+0000 7f55ed457700 1 --2- 172.21.15.117:0/3791437478 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e80aec90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f55dc002f30 tx=0x7f55dc006dc0).stop 2020-11-20T13:56:03.921 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.913+0000 7f55ed457700 1 -- 172.21.15.117:0/3791437478 shutdown_connections 2020-11-20T13:56:03.921 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.913+0000 7f55ed457700 1 --2- 172.21.15.117:0/3791437478 >> v2:172.21.15.85:3300/0 conn(0x7f55e8099fa0 0x7f55e809a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.921 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.913+0000 7f55ed457700 1 --2- 172.21.15.117:0/3791437478 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e80aec90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.921 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.913+0000 7f55ed457700 1 -- 172.21.15.117:0/3791437478 >> 172.21.15.117:0/3791437478 conn(0x7f55e805b9f0 msgr2=0x7f55e805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:03.922 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.913+0000 7f55ed457700 1 -- 172.21.15.117:0/3791437478 shutdown_connections 2020-11-20T13:56:03.922 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.913+0000 7f55ed457700 1 -- 172.21.15.117:0/3791437478 wait complete. 2020-11-20T13:56:03.924 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.914+0000 7f55ed457700 1 Processor -- start 2020-11-20T13:56:03.924 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.914+0000 7f55ed457700 1 -- start start 2020-11-20T13:56:03.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55e8099fa0 0x7f55e8090140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e8099050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e80b5860 0x7f55e8141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e809e960 con 0x7f55e8099fa0 2020-11-20T13:56:03.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8063ae0 con 0x7f55e80b5860 2020-11-20T13:56:03.926 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e809e420 con 0x7f55e80ae8b0 2020-11-20T13:56:03.926 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e80b5860 0x7f55e8141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.927 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e80b5860 0x7f55e8141cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40690/0 (socket says 172.21.15.117:40690) 2020-11-20T13:56:03.927 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 -- 172.21.15.117:0/4036621235 learned_addr learned my addr 172.21.15.117:0/4036621235 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:03.927 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e6ffd700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55e8099fa0 0x7f55e8090140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.927 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e67fc700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e8099050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.928 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 -- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 msgr2=0x7f55e8099050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.928 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e8099050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.928 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 -- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55e8099fa0 msgr2=0x7f55e8090140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:03.928 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55e8099fa0 0x7f55e8090140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:03.928 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 -- 172.21.15.117:0/4036621235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e809e6c0 con 0x7f55e80b5860 2020-11-20T13:56:03.929 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55e77fe700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e80b5860 0x7f55e8141cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f55e000e000 tx=0x7f55e000f040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.929 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55d7fff700 1 -- 172.21.15.117:0/4036621235 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55e0012040 con 0x7f55e80b5860 2020-11-20T13:56:03.929 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55d7fff700 1 -- 172.21.15.117:0/4036621235 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e000be40 con 0x7f55e80b5860 2020-11-20T13:56:03.929 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e809df10 con 0x7f55e80b5860 2020-11-20T13:56:03.930 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55d7fff700 1 -- 172.21.15.117:0/4036621235 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55e0015740 con 0x7f55e80b5860 2020-11-20T13:56:03.930 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.915+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e81423e0 con 0x7f55e80b5860 2020-11-20T13:56:03.930 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.916+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e809e220 con 0x7f55e80b5860 2020-11-20T13:56:03.930 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.916+0000 7f55d7fff700 1 -- 172.21.15.117:0/4036621235 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f55e00158a0 con 0x7f55e80b5860 2020-11-20T13:56:03.930 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.917+0000 7f55d7fff700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f55c40513c0 0x7f55c4053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:03.931 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.917+0000 7f55d7fff700 1 -- 172.21.15.117:0/4036621235 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f55e006bce0 con 0x7f55e80b5860 2020-11-20T13:56:03.931 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.917+0000 7f55e6ffd700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f55c40513c0 0x7f55c4053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:03.931 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.917+0000 7f55e6ffd700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f55c40513c0 0x7f55c4053880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f55dc01b000 tx=0x7f55dc015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:03.931 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:03.920+0000 7f55d7fff700 1 -- 172.21.15.117:0/4036621235 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f55e0043f00 con 0x7f55e80b5860 2020-11-20T13:56:04.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.119+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55e8057050 con 0x7f55e80b5860 2020-11-20T13:56:04.121 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.119+0000 7f55d7fff700 1 -- 172.21.15.117:0/4036621235 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7f55e0047520 con 0x7f55e80b5860 2020-11-20T13:56:04.121 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:04.121 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:55:51.676934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/1587825839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1587825839},{"type":"v1","addr":"172.21.15.117:6835","nonce":1587825839}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:04.123 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f55c40513c0 msgr2=0x7f55c4053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:04.123 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f55c40513c0 0x7f55c4053880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f55dc01b000 tx=0x7f55dc015000).stop 2020-11-20T13:56:04.123 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e80b5860 msgr2=0x7f55e8141cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:04.123 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e80b5860 0x7f55e8141cb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f55e000e000 tx=0x7f55e000f040).stop 2020-11-20T13:56:04.123 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 shutdown_connections 2020-11-20T13:56:04.124 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f55c40513c0 0x7f55c4053880 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:04.124 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e80b5860 0x7f55e8141cb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:04.124 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55e8099fa0 0x7f55e8090140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:04.124 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.120+0000 7f55ed457700 1 --2- 172.21.15.117:0/4036621235 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55e80ae8b0 0x7f55e8099050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:04.124 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.121+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 >> 172.21.15.117:0/4036621235 conn(0x7f55e805b9f0 msgr2=0x7f55e80b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:04.125 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.121+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 shutdown_connections 2020-11-20T13:56:04.125 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:04.121+0000 7f55ed457700 1 -- 172.21.15.117:0/4036621235 wait complete. 2020-11-20T13:56:04.125 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2020-11-20T13:56:04.333 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:55:50.675265+0000', 'modified': '2020-11-20T13:55:51.676934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/1587825839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1587825839}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1587825839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:04.334 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:04.334 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:04.462 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":4288,"want_state":"up:active","state":"up:active","rank_uptime":13.777276877,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":20.167202926000002} 2020-11-20T13:56:04.463 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 4288, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 13.777276877, "state": "up:active", "uptime": 20.167202926, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:04.463 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:04.464 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-20T13:56:04.464 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi117.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-20T13:56:04.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:04.464 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:04.481 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:04.583 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:04.584 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:04.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:04.612 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:04.667 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-20T13:56:04.667 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:04.711 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:04.738 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:04.738 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:04.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:2020-11-20T13:56:04.782+0000 7fe0487591c0 -1 init, newargv = 0x5576810a3580 newargc=9 2020-11-20T13:56:04.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[26196]: starting ceph client 2020-11-20T13:56:04.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[26196]: starting fuse 2020-11-20T13:56:04.807 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:04.807 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:04.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:04.808 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:04.861 INFO:teuthology.orchestra.run.smithi117.stdout:47 2020-11-20T13:56:04.862 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-20T13:56:04.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:04.862 INFO:teuthology.orchestra.run.smithi117:> sudo python3 -c ' 2020-11-20T13:56:04.863 INFO:teuthology.orchestra.run.smithi117:> import glob 2020-11-20T13:56:04.863 INFO:teuthology.orchestra.run.smithi117:> import re 2020-11-20T13:56:04.863 INFO:teuthology.orchestra.run.smithi117:> import os 2020-11-20T13:56:04.864 INFO:teuthology.orchestra.run.smithi117:> import subprocess 2020-11-20T13:56:04.865 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:04.866 INFO:teuthology.orchestra.run.smithi117:> def find_socket(client_name): 2020-11-20T13:56:04.866 INFO:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-20T13:56:04.866 INFO:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2020-11-20T13:56:04.866 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:04.867 INFO:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2020-11-20T13:56:04.867 INFO:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2020-11-20T13:56:04.868 INFO:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2020-11-20T13:56:04.868 INFO:teuthology.orchestra.run.smithi117:> return files[0] 2020-11-20T13:56:04.868 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:04.869 INFO:teuthology.orchestra.run.smithi117:> for f in files: 2020-11-20T13:56:04.869 INFO:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:04.869 INFO:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:04.870 INFO:teuthology.orchestra.run.smithi117:> return f 2020-11-20T13:56:04.870 INFO:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:04.870 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:04.871 INFO:teuthology.orchestra.run.smithi117:> print(find_socket("client.0")) 2020-11-20T13:56:04.871 INFO:teuthology.orchestra.run.smithi117:> ' 2020-11-20T13:56:04.953 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.0.26196.asok 2020-11-20T13:56:04.956 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.26196.asok 2020-11-20T13:56:04.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:04.957 INFO:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.26196.asok status 2020-11-20T13:56:05.063 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2020-11-20T13:56:05.063 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2020-11-20T13:56:05.063 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:05.063 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:05.064 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "0", 2020-11-20T13:56:05.064 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2020-11-20T13:56:05.064 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-20T13:56:05.064 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "26196", 2020-11-20T13:56:05.064 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2020-11-20T13:56:05.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:05.065 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2020-11-20T13:56:05.065 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:05.066 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 4663, 2020-11-20T13:56:05.066 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2020-11-20T13:56:05.066 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2020-11-20T13:56:05.066 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2020-11-20T13:56:05.067 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 4663 2020-11-20T13:56:05.067 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:05.067 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:05.067 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:05.067 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:05.068 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2617397902 2020-11-20T13:56:05.068 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:05.068 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:05.068 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:05.068 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:05.069 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:05.069 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2617397902 2020-11-20T13:56:05.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:05.069 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.4663 172.21.15.117:0/2617397902", 2020-11-20T13:56:05.070 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "172.21.15.117:0/2617397902", 2020-11-20T13:56:05.070 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2020-11-20T13:56:05.070 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 5, 2020-11-20T13:56:05.070 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 25, 2020-11-20T13:56:05.070 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:05.071 INFO:teuthology.orchestra.run.smithi117.stdout: "blacklisted": false 2020-11-20T13:56:05.071 INFO:teuthology.orchestra.run.smithi117.stdout:} 2020-11-20T13:56:05.072 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:05.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.176+0000 7fe2f3fe7700 1 Processor -- start 2020-11-20T13:56:05.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.177+0000 7fe2f3fe7700 1 -- start start 2020-11-20T13:56:05.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.177+0000 7fe2f3fe7700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec143d40 0x7fe2ec0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.177+0000 7fe2f3fe7700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec1441d0 con 0x7fe2ec1438e0 2020-11-20T13:56:05.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.177+0000 7fe2f3fe7700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec09b920 con 0x7fe2ec098e30 2020-11-20T13:56:05.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.177+0000 7fe2f3fe7700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec0b5510 con 0x7fe2ec143d40 2020-11-20T13:56:05.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f1d83700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe2ec098e30 0x7fe2ec09b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59874/0 (socket says 172.21.15.117:59874) 2020-11-20T13:56:05.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f1d83700 1 -- 172.21.15.117:0/1127901846 learned_addr learned my addr 172.21.15.117:0/1127901846 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:05.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f2584700 1 --2- 172.21.15.117:0/1127901846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec143d40 0x7fe2ec0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251017471 0 0) 0x7fe2ec09b920 con 0x7fe2ec098e30 2020-11-20T13:56:05.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d4003230 con 0x7fe2ec098e30 2020-11-20T13:56:05.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452512951 0 0) 0x7fe2e80010d0 con 0x7fe2ec1438e0 2020-11-20T13:56:05.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ec09b920 con 0x7fe2ec1438e0 2020-11-20T13:56:05.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f2584700 1 -- 172.21.15.117:0/1127901846 >> v1:172.21.15.117:6789/0 conn(0x7fe2ec098e30 legacy=0x7fe2ec09b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1819408050 0 0) 0x7fe2d4003230 con 0x7fe2ec098e30 2020-11-20T13:56:05.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f2584700 1 -- 172.21.15.117:0/1127901846 >> v1:172.21.15.85:6789/0 conn(0x7fe2ec1438e0 legacy=0x7fe2ec0ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f2584700 1 -- 172.21.15.117:0/1127901846 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0b57b0 con 0x7fe2ec143d40 2020-11-20T13:56:05.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f2584700 1 --2- 172.21.15.117:0/1127901846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec143d40 0x7fe2ec0aee00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe2dc001110 tx=0x7fe2e40091a0).ready entity=mon.2 client_cookie=15274d1939d0ad39 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2dc001710 con 0x7fe2ec143d40 2020-11-20T13:56:05.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e400b880 con 0x7fe2ec143d40 2020-11-20T13:56:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f0d81700 1 -- 172.21.15.117:0/1127901846 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2e400b9f0 con 0x7fe2ec143d40 2020-11-20T13:56:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/1127901846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec143d40 msgr2=0x7fe2ec0aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.178+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/1127901846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec143d40 0x7fe2ec0aee00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fe2dc001110 tx=0x7fe2e40091a0).stop 2020-11-20T13:56:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.179+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/1127901846 shutdown_connections 2020-11-20T13:56:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.179+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/1127901846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec143d40 0x7fe2ec0aee00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.179+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/1127901846 >> 172.21.15.117:0/1127901846 conn(0x7fe2ec05b9f0 msgr2=0x7fe2ec05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.179+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/1127901846 shutdown_connections 2020-11-20T13:56:05.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.179+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/1127901846 wait complete. 2020-11-20T13:56:05.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.180+0000 7fe2f3fe7700 1 Processor -- start 2020-11-20T13:56:05.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.180+0000 7fe2f3fe7700 1 -- start start 2020-11-20T13:56:05.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.180+0000 7fe2f3fe7700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2ec098e30 0x7fe2ec13e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f3fe7700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec1438e0 0x7fe2ec09f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f3fe7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec143d40 0x7fe2ec0b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f3fe7700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec09f7f0 con 0x7fe2ec098e30 2020-11-20T13:56:05.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f3fe7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec154150 con 0x7fe2ec143d40 2020-11-20T13:56:05.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f3fe7700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec154d50 con 0x7fe2ec1438e0 2020-11-20T13:56:05.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec143d40 0x7fe2ec0b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec143d40 0x7fe2ec0b88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40722/0 (socket says 172.21.15.117:40722) 2020-11-20T13:56:05.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 -- 172.21.15.117:0/2138029200 learned_addr learned my addr 172.21.15.117:0/2138029200 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:05.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f1d83700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2ec098e30 0x7fe2ec13e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f1582700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec1438e0 0x7fe2ec09f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 -- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec1438e0 msgr2=0x7fe2ec09f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec1438e0 0x7fe2ec09f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 -- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2ec098e30 msgr2=0x7fe2ec13e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2ec098e30 0x7fe2ec13e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 -- 172.21.15.117:0/2138029200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec154ff0 con 0x7fe2ec143d40 2020-11-20T13:56:05.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f1d83700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2ec098e30 0x7fe2ec13e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:05.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f2584700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec143d40 0x7fe2ec0b88f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fe2dc001110 tx=0x7fe2e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2daffd700 1 -- 172.21.15.117:0/2138029200 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2e4011db0 con 0x7fe2ec143d40 2020-11-20T13:56:05.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2daffd700 1 -- 172.21.15.117:0/2138029200 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e4020400 con 0x7fe2ec143d40 2020-11-20T13:56:05.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2daffd700 1 -- 172.21.15.117:0/2138029200 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2e4020760 con 0x7fe2ec143d40 2020-11-20T13:56:05.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec157a00 con 0x7fe2ec143d40 2020-11-20T13:56:05.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.181+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec0b8de0 con 0x7fe2ec143d40 2020-11-20T13:56:05.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.182+0000 7fe2daffd700 1 -- 172.21.15.117:0/2138029200 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe2e4027030 con 0x7fe2ec143d40 2020-11-20T13:56:05.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.182+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec057050 con 0x7fe2ec143d40 2020-11-20T13:56:05.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.182+0000 7fe2daffd700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe2cc051210 0x7fe2cc0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.183+0000 7fe2daffd700 1 -- 172.21.15.117:0/2138029200 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe2e4069dc0 con 0x7fe2ec143d40 2020-11-20T13:56:05.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.183+0000 7fe2f1d83700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe2cc051210 0x7fe2cc0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.183+0000 7fe2f1d83700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe2cc051210 0x7fe2cc0536d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe2dc00b000 tx=0x7fe2dc000b80).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.186+0000 7fe2daffd700 1 -- 172.21.15.117:0/2138029200 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe2e4041fe0 con 0x7fe2ec143d40 2020-11-20T13:56:05.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.336+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2ec155180 con 0x7fe2ec143d40 2020-11-20T13:56:05.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.337+0000 7fe2daffd700 1 -- 172.21.15.117:0/2138029200 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fe2e4015020 con 0x7fe2ec143d40 2020-11-20T13:56:05.338 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:05.339 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe2cc051210 msgr2=0x7fe2cc0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe2cc051210 0x7fe2cc0536d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe2dc00b000 tx=0x7fe2dc000b80).stop 2020-11-20T13:56:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec143d40 msgr2=0x7fe2ec0b88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec143d40 0x7fe2ec0b88f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fe2dc001110 tx=0x7fe2e4016000).stop 2020-11-20T13:56:05.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 shutdown_connections 2020-11-20T13:56:05.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe2cc051210 0x7fe2cc0536d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ec143d40 0x7fe2ec0b88f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe2ec098e30 0x7fe2ec13e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 --2- 172.21.15.117:0/2138029200 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe2ec1438e0 0x7fe2ec09f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.338+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 >> 172.21.15.117:0/2138029200 conn(0x7fe2ec05b9f0 msgr2=0x7fe2ec142b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.339+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 shutdown_connections 2020-11-20T13:56:05.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.339+0000 7fe2f3fe7700 1 -- 172.21.15.117:0/2138029200 wait complete. 2020-11-20T13:56:05.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:05.352 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:05.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 Processor -- start 2020-11-20T13:56:05.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 -- start start 2020-11-20T13:56:05.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f481009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f481009a110 0x7f48101437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4810143c90 0x7f48100b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48100999d0 con 0x7f481009a110 2020-11-20T13:56:05.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4810099c70 con 0x7f4810143c90 2020-11-20T13:56:05.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f48163f8700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4810144120 con 0x7f481009bd80 2020-11-20T13:56:05.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.460+0000 7f4814995700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4810143c90 0x7f48100b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4810143c90 0x7f48100b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40726/0 (socket says 172.21.15.117:40726) 2020-11-20T13:56:05.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 -- 172.21.15.117:0/854058208 learned_addr learned my addr 172.21.15.117:0/854058208 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:05.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f480ffff700 1 --2- 172.21.15.117:0/854058208 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f481009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f480f7fe700 1 --2- 172.21.15.117:0/854058208 >> v2:172.21.15.85:3300/0 conn(0x7f481009a110 0x7f48101437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 -- 172.21.15.117:0/854058208 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 msgr2=0x7f481009cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:05.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 --2- 172.21.15.117:0/854058208 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f481009cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 -- 172.21.15.117:0/854058208 >> v2:172.21.15.85:3300/0 conn(0x7f481009a110 msgr2=0x7f48101437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:05.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 --2- 172.21.15.117:0/854058208 >> v2:172.21.15.85:3300/0 conn(0x7f481009a110 0x7f48101437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 -- 172.21.15.117:0/854058208 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481009c160 con 0x7f4810143c90 2020-11-20T13:56:05.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f4814995700 1 --2- 172.21.15.117:0/854058208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810143c90 0x7f48100b4110 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f48080032b0 tx=0x7f4808009270).ready entity=mon.1 client_cookie=2968fd5f3201883c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f480effd700 1 -- 172.21.15.117:0/854058208 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f480800bc00 con 0x7f4810143c90 2020-11-20T13:56:05.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f480effd700 1 -- 172.21.15.117:0/854058208 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480800bd60 con 0x7f4810143c90 2020-11-20T13:56:05.478 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f480effd700 1 -- 172.21.15.117:0/854058208 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4808011400 con 0x7f4810143c90 2020-11-20T13:56:05.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f48163f8700 1 -- 172.21.15.117:0/854058208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810143c90 msgr2=0x7f48100b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.461+0000 7f48163f8700 1 --2- 172.21.15.117:0/854058208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810143c90 0x7f48100b4110 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f48080032b0 tx=0x7f4808009270).stop 2020-11-20T13:56:05.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.462+0000 7f48163f8700 1 -- 172.21.15.117:0/854058208 shutdown_connections 2020-11-20T13:56:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.462+0000 7f48163f8700 1 --2- 172.21.15.117:0/854058208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4810143c90 0x7f48100b4110 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.462+0000 7f48163f8700 1 --2- 172.21.15.117:0/854058208 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f481009cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.462+0000 7f48163f8700 1 --2- 172.21.15.117:0/854058208 >> v2:172.21.15.85:3300/0 conn(0x7f481009a110 0x7f48101437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.462+0000 7f48163f8700 1 -- 172.21.15.117:0/854058208 >> 172.21.15.117:0/854058208 conn(0x7f48100b9770 msgr2=0x7f48100b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.462+0000 7f48163f8700 1 -- 172.21.15.117:0/854058208 shutdown_connections 2020-11-20T13:56:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.462+0000 7f48163f8700 1 -- 172.21.15.117:0/854058208 wait complete. 2020-11-20T13:56:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 Processor -- start 2020-11-20T13:56:05.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 -- start start 2020-11-20T13:56:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481009a110 0x7f4810156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f48100b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4810143c90 0x7f4810159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48101467c0 con 0x7f4810143c90 2020-11-20T13:56:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4810146a30 con 0x7f481009a110 2020-11-20T13:56:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f48163f8700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4810146cd0 con 0x7f481009bd80 2020-11-20T13:56:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f480ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481009a110 0x7f4810156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f480ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481009a110 0x7f4810156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40732/0 (socket says 172.21.15.117:40732) 2020-11-20T13:56:05.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.463+0000 7f480ffff700 1 -- 172.21.15.117:0/348615499 learned_addr learned my addr 172.21.15.117:0/348615499 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:05.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480f7fe700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f48100b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f4814995700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4810143c90 0x7f4810159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480ffff700 1 -- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 msgr2=0x7f48100b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480ffff700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f48100b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480ffff700 1 -- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4810143c90 msgr2=0x7f4810159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480ffff700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4810143c90 0x7f4810159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.492 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480ffff700 1 -- 172.21.15.117:0/348615499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810149250 con 0x7f481009a110 2020-11-20T13:56:05.492 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f4814995700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4810143c90 0x7f4810159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:05.492 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480ffff700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481009a110 0x7f4810156140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f4800003070 tx=0x7f4800008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.493 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480d7fa700 1 -- 172.21.15.117:0/348615499 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4800009980 con 0x7f481009a110 2020-11-20T13:56:05.493 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480d7fa700 1 -- 172.21.15.117:0/348615499 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4800009ae0 con 0x7f481009a110 2020-11-20T13:56:05.493 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f480d7fa700 1 -- 172.21.15.117:0/348615499 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4800009e40 con 0x7f481009a110 2020-11-20T13:56:05.493 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4810147170 con 0x7f481009a110 2020-11-20T13:56:05.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.464+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48101464c0 con 0x7f481009a110 2020-11-20T13:56:05.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.465+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810146e90 con 0x7f481009a110 2020-11-20T13:56:05.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.465+0000 7f480d7fa700 1 -- 172.21.15.117:0/348615499 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4800013480 con 0x7f481009a110 2020-11-20T13:56:05.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.465+0000 7f480d7fa700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47f0051290 0x7f47f0053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.465+0000 7f480d7fa700 1 -- 172.21.15.117:0/348615499 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f4800069980 con 0x7f481009a110 2020-11-20T13:56:05.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.465+0000 7f480f7fe700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47f0051290 0x7f47f0053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.466+0000 7f480f7fe700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47f0051290 0x7f47f0053750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f480400c000 tx=0x7f4804008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.469+0000 7f480d7fa700 1 -- 172.21.15.117:0/348615499 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f4800041b40 con 0x7f481009a110 2020-11-20T13:56:05.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.617+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4810057050 con 0x7f481009a110 2020-11-20T13:56:05.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.617+0000 7f480d7fa700 1 -- 172.21.15.117:0/348615499 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4800045160 con 0x7f481009a110 2020-11-20T13:56:05.619 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:05.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.618+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47f0051290 msgr2=0x7f47f0053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.618+0000 7f48163f8700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47f0051290 0x7f47f0053750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f480400c000 tx=0x7f4804008040).stop 2020-11-20T13:56:05.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.618+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481009a110 msgr2=0x7f4810156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.618+0000 7f48163f8700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481009a110 0x7f4810156140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f4800003070 tx=0x7f4800008040).stop 2020-11-20T13:56:05.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 shutdown_connections 2020-11-20T13:56:05.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f47f0051290 0x7f47f0053750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.621 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4810143c90 0x7f4810159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481009a110 0x7f4810156140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 --2- 172.21.15.117:0/348615499 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f481009bd80 0x7f48100b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 >> 172.21.15.117:0/348615499 conn(0x7f48100b9770 msgr2=0x7f481005ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 shutdown_connections 2020-11-20T13:56:05.622 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.619+0000 7f48163f8700 1 -- 172.21.15.117:0/348615499 wait complete. 2020-11-20T13:56:05.630 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:05.740 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.739+0000 7f34a1487700 1 Processor -- start 2020-11-20T13:56:05.741 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.739+0000 7f34a1487700 1 -- start start 2020-11-20T13:56:05.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f34a1487700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f349c09cb40 0x7f349c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f34a1487700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349c0999d0 con 0x7f349c143c90 2020-11-20T13:56:05.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f34a1487700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349c099c70 con 0x7f349c09bd80 2020-11-20T13:56:05.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f34a1487700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c0997f0 con 0x7f349c09cb40 2020-11-20T13:56:05.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f349a7fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f349c09cb40 0x7f349c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f349affd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f349c09bd80 0x7f349c09a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59900/0 (socket says 172.21.15.117:59900) 2020-11-20T13:56:05.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f349affd700 1 -- 172.21.15.117:0/3285249263 learned_addr learned my addr 172.21.15.117:0/3285249263 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:05.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f349a7fc700 1 -- 172.21.15.117:0/3285249263 >> v1:172.21.15.117:6789/0 conn(0x7f349c09bd80 legacy=0x7f349c09a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f349a7fc700 1 -- 172.21.15.117:0/3285249263 >> v1:172.21.15.85:6789/0 conn(0x7f349c143c90 legacy=0x7f349c0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f349a7fc700 1 -- 172.21.15.117:0/3285249263 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c144120 con 0x7f349c09cb40 2020-11-20T13:56:05.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f3499ffb700 1 -- 172.21.15.117:0/3285249263 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872453971 0 0) 0x7f349c099c70 con 0x7f349c09bd80 2020-11-20T13:56:05.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.740+0000 7f349a7fc700 1 --2- 172.21.15.117:0/3285249263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09cb40 0x7f349c1437a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f348c002f30 tx=0x7f348c006dc0).ready entity=mon.1 client_cookie=fdf104933d06a3c1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f3499ffb700 1 -- 172.21.15.117:0/3285249263 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c0097f0 con 0x7f349c09cb40 2020-11-20T13:56:05.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f3499ffb700 1 -- 172.21.15.117:0/3285249263 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c009950 con 0x7f349c09cb40 2020-11-20T13:56:05.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f3499ffb700 1 -- 172.21.15.117:0/3285249263 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c009cb0 con 0x7f349c09cb40 2020-11-20T13:56:05.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f34a1487700 1 -- 172.21.15.117:0/3285249263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09cb40 msgr2=0x7f349c1437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f34a1487700 1 --2- 172.21.15.117:0/3285249263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09cb40 0x7f349c1437a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f348c002f30 tx=0x7f348c006dc0).stop 2020-11-20T13:56:05.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f34a1487700 1 -- 172.21.15.117:0/3285249263 shutdown_connections 2020-11-20T13:56:05.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f34a1487700 1 --2- 172.21.15.117:0/3285249263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09cb40 0x7f349c1437a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f34a1487700 1 -- 172.21.15.117:0/3285249263 >> 172.21.15.117:0/3285249263 conn(0x7f349c0b9770 msgr2=0x7f349c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f34a1487700 1 -- 172.21.15.117:0/3285249263 shutdown_connections 2020-11-20T13:56:05.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.741+0000 7f34a1487700 1 -- 172.21.15.117:0/3285249263 wait complete. 2020-11-20T13:56:05.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.742+0000 7f34a1487700 1 Processor -- start 2020-11-20T13:56:05.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.742+0000 7f34a1487700 1 -- start start 2020-11-20T13:56:05.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09bd80 0x7f349c149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.750 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f349c09cb40 0x7f349c1403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f349c143c90 0x7f349c140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c154550 con 0x7f349c09cb40 2020-11-20T13:56:05.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c1547f0 con 0x7f349c09bd80 2020-11-20T13:56:05.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c154a90 con 0x7f349c143c90 2020-11-20T13:56:05.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09bd80 0x7f349c149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09bd80 0x7f349c149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40744/0 (socket says 172.21.15.117:40744) 2020-11-20T13:56:05.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 -- 172.21.15.117:0/3717729244 learned_addr learned my addr 172.21.15.117:0/3717729244 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:05.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349a7fc700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f349c09cb40 0x7f349c1403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349b7fe700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f349c143c90 0x7f349c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 -- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f349c143c90 msgr2=0x7f349c140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f349c143c90 0x7f349c140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 -- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f349c09cb40 msgr2=0x7f349c1403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f349c09cb40 0x7f349c1403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 -- 172.21.15.117:0/3717729244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c154f30 con 0x7f349c09bd80 2020-11-20T13:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349a7fc700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f349c09cb40 0x7f349c1403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349b7fe700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f349c143c90 0x7f349c140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f349affd700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09bd80 0x7f349c149df0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f3490000b80 tx=0x7f34900079b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f348bfff700 1 -- 172.21.15.117:0/3717729244 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34900072e0 con 0x7f349c09bd80 2020-11-20T13:56:05.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f348bfff700 1 -- 172.21.15.117:0/3717729244 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490007440 con 0x7f349c09bd80 2020-11-20T13:56:05.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c0b3390 con 0x7f349c09bd80 2020-11-20T13:56:05.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f348bfff700 1 -- 172.21.15.117:0/3717729244 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34900057d0 con 0x7f349c09bd80 2020-11-20T13:56:05.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.743+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c154010 con 0x7f349c09bd80 2020-11-20T13:56:05.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.744+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c154c50 con 0x7f349c09bd80 2020-11-20T13:56:05.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.744+0000 7f348bfff700 1 -- 172.21.15.117:0/3717729244 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3490010070 con 0x7f349c09bd80 2020-11-20T13:56:05.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.745+0000 7f348bfff700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3478051320 0x7f34780537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:05.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.745+0000 7f348bfff700 1 -- 172.21.15.117:0/3717729244 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f34900687a0 con 0x7f349c09bd80 2020-11-20T13:56:05.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.745+0000 7f349a7fc700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3478051320 0x7f34780537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:05.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.745+0000 7f349a7fc700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3478051320 0x7f34780537e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f348c016000 tx=0x7f348c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:05.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.748+0000 7f348bfff700 1 -- 172.21.15.117:0/3717729244 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f34900409c0 con 0x7f349c09bd80 2020-11-20T13:56:05.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.898+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f349c057050 con 0x7f349c09bd80 2020-11-20T13:56:05.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.899+0000 7f348bfff700 1 -- 172.21.15.117:0/3717729244 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3490043fe0 con 0x7f349c09bd80 2020-11-20T13:56:05.901 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:05.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3478051320 msgr2=0x7f34780537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3478051320 0x7f34780537e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f348c016000 tx=0x7f348c015000).stop 2020-11-20T13:56:05.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09bd80 msgr2=0x7f349c149df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:05.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09bd80 0x7f349c149df0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f3490000b80 tx=0x7f34900079b0).stop 2020-11-20T13:56:05.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 shutdown_connections 2020-11-20T13:56:05.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3478051320 0x7f34780537e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f349c143c90 0x7f349c140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f349c09bd80 0x7f349c149df0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 --2- 172.21.15.117:0/3717729244 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f349c09cb40 0x7f349c1403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 >> 172.21.15.117:0/3717729244 conn(0x7f349c0b9770 msgr2=0x7f349c0b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 shutdown_connections 2020-11-20T13:56:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:05.900+0000 7f34a1487700 1 -- 172.21.15.117:0/3717729244 wait complete. 2020-11-20T13:56:05.914 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:06.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.025+0000 7fc693df3700 1 Processor -- start 2020-11-20T13:56:06.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.025+0000 7fc693df3700 1 -- start start 2020-11-20T13:56:06.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc693df3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c09cb40 0x7fc68c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc693df3700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc68c143c90 0x7fc68c0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc693df3700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68c0999d0 con 0x7fc68c09bd80 2020-11-20T13:56:06.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc693df3700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68c0997f0 con 0x7fc68c143c90 2020-11-20T13:56:06.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc693df3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c099c70 con 0x7fc68c09cb40 2020-11-20T13:56:06.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc692390700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc68c143c90 0x7fc68c0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc691b8f700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc68c09bd80 0x7fc68c09a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59912/0 (socket says 172.21.15.117:59912) 2020-11-20T13:56:06.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc691b8f700 1 -- 172.21.15.117:0/2603599744 learned_addr learned my addr 172.21.15.117:0/2603599744 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:06.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc69138e700 1 --2- 172.21.15.117:0/2603599744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c09cb40 0x7fc68c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc692390700 1 -- 172.21.15.117:0/2603599744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c09cb40 msgr2=0x7fc68c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:06.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc692390700 1 --2- 172.21.15.117:0/2603599744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c09cb40 0x7fc68c1437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc692390700 1 -- 172.21.15.117:0/2603599744 >> v1:172.21.15.117:6789/0 conn(0x7fc68c09bd80 legacy=0x7fc68c09a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc690b8d700 1 -- 172.21.15.117:0/2603599744 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571412250 0 0) 0x7fc68c0999d0 con 0x7fc68c09bd80 2020-11-20T13:56:06.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc692390700 1 -- 172.21.15.117:0/2603599744 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c144120 con 0x7fc68c143c90 2020-11-20T13:56:06.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc692390700 1 --2- 172.21.15.117:0/2603599744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c143c90 0x7fc68c0b4110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fc68c09b9c0 tx=0x7fc6840091a0).ready entity=mon.1 client_cookie=9512c128b046354f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc690b8d700 1 -- 172.21.15.117:0/2603599744 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc684009e70 con 0x7fc68c143c90 2020-11-20T13:56:06.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc690b8d700 1 -- 172.21.15.117:0/2603599744 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68400bbc0 con 0x7fc68c143c90 2020-11-20T13:56:06.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc690b8d700 1 -- 172.21.15.117:0/2603599744 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc68400bd30 con 0x7fc68c143c90 2020-11-20T13:56:06.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc693df3700 1 -- 172.21.15.117:0/2603599744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c143c90 msgr2=0x7fc68c0b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.026+0000 7fc693df3700 1 --2- 172.21.15.117:0/2603599744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c143c90 0x7fc68c0b4110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fc68c09b9c0 tx=0x7fc6840091a0).stop 2020-11-20T13:56:06.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.027+0000 7fc693df3700 1 -- 172.21.15.117:0/2603599744 shutdown_connections 2020-11-20T13:56:06.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.027+0000 7fc693df3700 1 --2- 172.21.15.117:0/2603599744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c143c90 0x7fc68c0b4110 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.027+0000 7fc693df3700 1 --2- 172.21.15.117:0/2603599744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c09cb40 0x7fc68c1437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.027+0000 7fc693df3700 1 -- 172.21.15.117:0/2603599744 >> 172.21.15.117:0/2603599744 conn(0x7fc68c0b9770 msgr2=0x7fc68c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:06.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.027+0000 7fc693df3700 1 -- 172.21.15.117:0/2603599744 shutdown_connections 2020-11-20T13:56:06.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.027+0000 7fc693df3700 1 -- 172.21.15.117:0/2603599744 wait complete. 2020-11-20T13:56:06.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.028+0000 7fc693df3700 1 Processor -- start 2020-11-20T13:56:06.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.028+0000 7fc693df3700 1 -- start start 2020-11-20T13:56:06.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c09bd80 0x7fc68c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc68c09cb40 0x7fc68c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c143c90 0x7fc68c15a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c140b10 con 0x7fc68c09cb40 2020-11-20T13:56:06.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c156880 con 0x7fc68c09bd80 2020-11-20T13:56:06.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c156aa0 con 0x7fc68c143c90 2020-11-20T13:56:06.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c09bd80 0x7fc68c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c09bd80 0x7fc68c156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40758/0 (socket says 172.21.15.117:40758) 2020-11-20T13:56:06.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 -- 172.21.15.117:0/1929286639 learned_addr learned my addr 172.21.15.117:0/1929286639 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:06.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc69138e700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc68c09cb40 0x7fc68c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc692390700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c143c90 0x7fc68c15a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 -- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c143c90 msgr2=0x7fc68c15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c143c90 0x7fc68c15a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 -- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc68c09cb40 msgr2=0x7fc68c0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc68c09cb40 0x7fc68c0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 -- 172.21.15.117:0/1929286639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c156d40 con 0x7fc68c09bd80 2020-11-20T13:56:06.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc691b8f700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c09bd80 0x7fc68c156390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fc68c140c50 tx=0x7fc6880079b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc67affd700 1 -- 172.21.15.117:0/1929286639 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6880072e0 con 0x7fc68c09bd80 2020-11-20T13:56:06.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc67affd700 1 -- 172.21.15.117:0/1929286639 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc688007440 con 0x7fc68c09bd80 2020-11-20T13:56:06.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c156fe0 con 0x7fc68c09bd80 2020-11-20T13:56:06.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc67affd700 1 -- 172.21.15.117:0/1929286639 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6880057d0 con 0x7fc68c09bd80 2020-11-20T13:56:06.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.029+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c159960 con 0x7fc68c09bd80 2020-11-20T13:56:06.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.030+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68c057050 con 0x7fc68c09bd80 2020-11-20T13:56:06.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.030+0000 7fc67affd700 1 -- 172.21.15.117:0/1929286639 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc688010070 con 0x7fc68c09bd80 2020-11-20T13:56:06.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.031+0000 7fc67affd700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc66c0513e0 0x7fc66c0538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.031+0000 7fc67affd700 1 -- 172.21.15.117:0/1929286639 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc688068820 con 0x7fc68c09bd80 2020-11-20T13:56:06.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.031+0000 7fc69138e700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc66c0513e0 0x7fc66c0538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.031+0000 7fc69138e700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc66c0513e0 0x7fc66c0538a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fc67c00c000 tx=0x7fc67c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.034+0000 7fc67affd700 1 -- 172.21.15.117:0/1929286639 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc688040940 con 0x7fc68c09bd80 2020-11-20T13:56:06.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.183+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc68c143190 con 0x7fc68c09bd80 2020-11-20T13:56:06.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.183+0000 7fc67affd700 1 -- 172.21.15.117:0/1929286639 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc688043f60 con 0x7fc68c09bd80 2020-11-20T13:56:06.185 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.184+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc66c0513e0 msgr2=0x7fc66c0538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc66c0513e0 0x7fc66c0538a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fc67c00c000 tx=0x7fc67c008040).stop 2020-11-20T13:56:06.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c09bd80 msgr2=0x7fc68c156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c09bd80 0x7fc68c156390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fc68c140c50 tx=0x7fc6880079b0).stop 2020-11-20T13:56:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 shutdown_connections 2020-11-20T13:56:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc66c0513e0 0x7fc66c0538a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc68c143c90 0x7fc68c15a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68c09bd80 0x7fc68c156390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 --2- 172.21.15.117:0/1929286639 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc68c09cb40 0x7fc68c0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 >> 172.21.15.117:0/1929286639 conn(0x7fc68c0b9770 msgr2=0x7fc68c09f090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:06.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 shutdown_connections 2020-11-20T13:56:06.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.185+0000 7fc693df3700 1 -- 172.21.15.117:0/1929286639 wait complete. 2020-11-20T13:56:06.196 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:06.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.304+0000 7f9916c8d700 1 Processor -- start 2020-11-20T13:56:06.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.304+0000 7f9916c8d700 1 -- start start 2020-11-20T13:56:06.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f9916c8d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f991009ca70 0x7f9910060400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f9916c8d700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9910060fb0 con 0x7f99100b01d0 2020-11-20T13:56:06.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f9916c8d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9910061250 con 0x7f9910145500 2020-11-20T13:56:06.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f9916c8d700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9910060dd0 con 0x7f991009ca70 2020-11-20T13:56:06.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f9907fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9910145500 0x7f9910145ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59928/0 (socket says 172.21.15.117:59928) 2020-11-20T13:56:06.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f9907fff700 1 -- 172.21.15.117:0/3129437685 learned_addr learned my addr 172.21.15.117:0/3129437685 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:06.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238660714 0 0) 0x7f9910061250 con 0x7f9910145500 2020-11-20T13:56:06.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f9914a29700 1 --2- 172.21.15.117:0/3129437685 >> v2:172.21.15.85:3300/0 conn(0x7f991009ca70 0x7f9910060400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98fc003230 con 0x7f9910145500 2020-11-20T13:56:06.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1184947781 0 0) 0x7f98fc003230 con 0x7f9910145500 2020-11-20T13:56:06.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 >> v2:172.21.15.85:3300/0 conn(0x7f991009ca70 msgr2=0x7f9910060400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:06.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 --2- 172.21.15.117:0/3129437685 >> v2:172.21.15.85:3300/0 conn(0x7f991009ca70 0x7f9910060400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 >> v1:172.21.15.85:6789/0 conn(0x7f99100b01d0 legacy=0x7f99100b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991009ce50 con 0x7f9910145500 2020-11-20T13:56:06.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f99000016d0 con 0x7f9910145500 2020-11-20T13:56:06.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 >> v1:172.21.15.117:6789/0 conn(0x7f9910145500 legacy=0x7f9910145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc003720 0x7f98fc003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98fc0044d0 0x7f98fc004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98fc004e40 0x7f98fc00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910060dd0 con 0x7f98fc003720 2020-11-20T13:56:06.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc004340 con 0x7f98fc004e40 2020-11-20T13:56:06.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.305+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc00bc40 con 0x7f98fc0044d0 2020-11-20T13:56:06.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98fc004e40 0x7f98fc00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f9914a29700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc003720 0x7f98fc003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f9907fff700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98fc0044d0 0x7f98fc004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 -- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98fc0044d0 msgr2=0x7f98fc004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:06.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98fc0044d0 0x7f98fc004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 -- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc003720 msgr2=0x7f98fc003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:06.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc003720 0x7f98fc003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 -- 172.21.15.117:0/3129437685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc003230 con 0x7f98fc004e40 2020-11-20T13:56:06.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 -- 172.21.15.117:0/3129437685 reap_dead start 2020-11-20T13:56:06.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f991522a700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98fc004e40 0x7f98fc00b720 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f99100b00b0 tx=0x7f9908007e50).ready entity=mon.1 client_cookie=76761da1af503896 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f990800c960 con 0x7f98fc004e40 2020-11-20T13:56:06.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800cac0 con 0x7f98fc004e40 2020-11-20T13:56:06.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f99077fe700 1 -- 172.21.15.117:0/3129437685 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f990800ce20 con 0x7f98fc004e40 2020-11-20T13:56:06.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f9916c8d700 1 -- 172.21.15.117:0/3129437685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98fc004e40 msgr2=0x7f98fc00b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.306+0000 7f9916c8d700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98fc004e40 0x7f98fc00b720 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f99100b00b0 tx=0x7f9908007e50).stop 2020-11-20T13:56:06.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.307+0000 7f9916c8d700 1 -- 172.21.15.117:0/3129437685 shutdown_connections 2020-11-20T13:56:06.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.307+0000 7f9916c8d700 1 --2- 172.21.15.117:0/3129437685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98fc004e40 0x7f98fc00b720 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.307+0000 7f9916c8d700 1 -- 172.21.15.117:0/3129437685 >> 172.21.15.117:0/3129437685 conn(0x7f99100ba890 msgr2=0x7f99100baca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:06.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.307+0000 7f9916c8d700 1 -- 172.21.15.117:0/3129437685 shutdown_connections 2020-11-20T13:56:06.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.307+0000 7f9916c8d700 1 -- 172.21.15.117:0/3129437685 wait complete. 2020-11-20T13:56:06.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.307+0000 7f9916c8d700 1 Processor -- start 2020-11-20T13:56:06.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9916c8d700 1 -- start start 2020-11-20T13:56:06.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9916c8d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc004e40 0x7f99100a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9916c8d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9910063800 0x7f9910153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9916c8d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99100b4430 0x7f9910064310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9916c8d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910154200 con 0x7f98fc004e40 2020-11-20T13:56:06.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9916c8d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99100b3750 con 0x7f99100b4430 2020-11-20T13:56:06.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9916c8d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100b39f0 con 0x7f9910063800 2020-11-20T13:56:06.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f991522a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99100b4430 0x7f9910064310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f991522a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99100b4430 0x7f9910064310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40782/0 (socket says 172.21.15.117:40782) 2020-11-20T13:56:06.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f991522a700 1 -- 172.21.15.117:0/373702417 learned_addr learned my addr 172.21.15.117:0/373702417 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:06.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9914a29700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc004e40 0x7f99100a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.308+0000 7f9907fff700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9910063800 0x7f9910153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f991522a700 1 -- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9910063800 msgr2=0x7f9910153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f991522a700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9910063800 0x7f9910153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f991522a700 1 -- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc004e40 msgr2=0x7f99100a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f991522a700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc004e40 0x7f99100a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f991522a700 1 -- 172.21.15.117:0/373702417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100b3c90 con 0x7f99100b4430 2020-11-20T13:56:06.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f991522a700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99100b4430 0x7f9910064310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9910157380 tx=0x7f9908015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f9905ffb700 1 -- 172.21.15.117:0/373702417 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9908007b80 con 0x7f99100b4430 2020-11-20T13:56:06.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f9905ffb700 1 -- 172.21.15.117:0/373702417 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990801d610 con 0x7f99100b4430 2020-11-20T13:56:06.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9910064800 con 0x7f99100b4430 2020-11-20T13:56:06.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f9905ffb700 1 -- 172.21.15.117:0/373702417 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f990801d970 con 0x7f99100b4430 2020-11-20T13:56:06.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.309+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9910064c70 con 0x7f99100b4430 2020-11-20T13:56:06.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.310+0000 7f9905ffb700 1 -- 172.21.15.117:0/373702417 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9908024030 con 0x7f99100b4430 2020-11-20T13:56:06.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.310+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9910057050 con 0x7f99100b4430 2020-11-20T13:56:06.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.310+0000 7f9905ffb700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98f00513c0 0x7f98f0053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.310+0000 7f9905ffb700 1 -- 172.21.15.117:0/373702417 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f990806a230 con 0x7f99100b4430 2020-11-20T13:56:06.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.310+0000 7f9914a29700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98f00513c0 0x7f98f0053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.311+0000 7f9914a29700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98f00513c0 0x7f98f0053880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f990c008000 tx=0x7f990c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.314+0000 7f9905ffb700 1 -- 172.21.15.117:0/373702417 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9908042340 con 0x7f99100b4430 2020-11-20T13:56:06.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.468+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f991006a7f0 con 0x7f99100b4430 2020-11-20T13:56:06.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.468+0000 7f9905ffb700 1 -- 172.21.15.117:0/373702417 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9908022020 con 0x7f99100b4430 2020-11-20T13:56:06.470 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:06.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98f00513c0 msgr2=0x7f98f0053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98f00513c0 0x7f98f0053880 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f990c008000 tx=0x7f990c008040).stop 2020-11-20T13:56:06.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99100b4430 msgr2=0x7f9910064310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99100b4430 0x7f9910064310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9910157380 tx=0x7f9908015000).stop 2020-11-20T13:56:06.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 shutdown_connections 2020-11-20T13:56:06.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98f00513c0 0x7f98f0053880 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99100b4430 0x7f9910064310 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98fc004e40 0x7f99100a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 --2- 172.21.15.117:0/373702417 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9910063800 0x7f9910153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 >> 172.21.15.117:0/373702417 conn(0x7f99100ba890 msgr2=0x7f99100b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:06.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 shutdown_connections 2020-11-20T13:56:06.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.470+0000 7f9916c8d700 1 -- 172.21.15.117:0/373702417 wait complete. 2020-11-20T13:56:06.481 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:06.482 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:06.587 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.586+0000 7fad5e246700 1 Processor -- start 2020-11-20T13:56:06.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.586+0000 7fad5e246700 1 -- start start 2020-11-20T13:56:06.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad5e246700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fad58098e30 0x7fad5809b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad5e246700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad5e246700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad580b5510 con 0x7fad5809b700 2020-11-20T13:56:06.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad5e246700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad580b57b0 con 0x7fad58098e30 2020-11-20T13:56:06.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad5e246700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580b5a50 con 0x7fad580ae6f0 2020-11-20T13:56:06.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad57fff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad577fe700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fad58098e30 0x7fad5809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad57fff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:40852/0 (socket says 172.21.15.117:40852) 2020-11-20T13:56:06.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad577fe700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fad58098e30 0x7fad5809b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:52412/0 (socket says 172.21.15.117:52412) 2020-11-20T13:56:06.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad57fff700 1 -- 172.21.15.117:0/2732029822 learned_addr learned my addr 172.21.15.117:0/2732029822 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:06.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.587+0000 7fad56ffd700 1 --1- 172.21.15.117:0/2732029822 >> v1:172.21.15.85:6789/0 conn(0x7fad5809b700 0x7fad581438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:43850/0 (socket says 172.21.15.117:43850) 2020-11-20T13:56:06.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad57fff700 1 -- 172.21.15.117:0/2732029822 >> v2:172.21.15.85:3300/0 conn(0x7fad58098e30 msgr2=0x7fad5809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:06.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad57fff700 1 --2- 172.21.15.117:0/2732029822 >> v2:172.21.15.85:3300/0 conn(0x7fad58098e30 0x7fad5809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad567fc700 1 -- 172.21.15.117:0/2732029822 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464977523 0 0) 0x7fad580b5510 con 0x7fad5809b700 2020-11-20T13:56:06.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad57fff700 1 -- 172.21.15.117:0/2732029822 >> v1:172.21.15.85:6789/0 conn(0x7fad5809b700 legacy=0x7fad581438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad57fff700 1 -- 172.21.15.117:0/2732029822 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58099210 con 0x7fad580ae6f0 2020-11-20T13:56:06.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad57fff700 1 --2- 172.21.15.117:0/2732029822 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580aeb60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fad500032b0 tx=0x7fad50009270).ready entity=mon.2 client_cookie=c54c7b4eab382b0f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad567fc700 1 -- 172.21.15.117:0/2732029822 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad50009e60 con 0x7fad580ae6f0 2020-11-20T13:56:06.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad567fc700 1 -- 172.21.15.117:0/2732029822 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5000bc00 con 0x7fad580ae6f0 2020-11-20T13:56:06.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad567fc700 1 -- 172.21.15.117:0/2732029822 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5000bd70 con 0x7fad580ae6f0 2020-11-20T13:56:06.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad5e246700 1 -- 172.21.15.117:0/2732029822 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 msgr2=0x7fad580aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.588+0000 7fad5e246700 1 --2- 172.21.15.117:0/2732029822 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580aeb60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fad500032b0 tx=0x7fad50009270).stop 2020-11-20T13:56:06.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.589+0000 7fad5e246700 1 -- 172.21.15.117:0/2732029822 shutdown_connections 2020-11-20T13:56:06.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.589+0000 7fad5e246700 1 --2- 172.21.15.117:0/2732029822 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580aeb60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.589+0000 7fad5e246700 1 --2- 172.21.15.117:0/2732029822 >> v2:172.21.15.85:3300/0 conn(0x7fad58098e30 0x7fad5809b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.589+0000 7fad5e246700 1 -- 172.21.15.117:0/2732029822 >> 172.21.15.117:0/2732029822 conn(0x7fad5805b9f0 msgr2=0x7fad5805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:06.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.589+0000 7fad5e246700 1 -- 172.21.15.117:0/2732029822 shutdown_connections 2020-11-20T13:56:06.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.589+0000 7fad5e246700 1 -- 172.21.15.117:0/2732029822 wait complete. 2020-11-20T13:56:06.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 Processor -- start 2020-11-20T13:56:06.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 -- start start 2020-11-20T13:56:06.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad58098e30 0x7fad58156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fad5809b700 0x7fad580b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580af160 con 0x7fad5809b700 2020-11-20T13:56:06.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580af400 con 0x7fad58098e30 2020-11-20T13:56:06.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.590+0000 7fad5e246700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580af6a0 con 0x7fad580ae6f0 2020-11-20T13:56:06.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad58098e30 0x7fad58156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad58098e30 0x7fad58156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40790/0 (socket says 172.21.15.117:40790) 2020-11-20T13:56:06.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 -- 172.21.15.117:0/1212618398 learned_addr learned my addr 172.21.15.117:0/1212618398 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:06.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad57fff700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad56ffd700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fad5809b700 0x7fad580b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 -- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 msgr2=0x7fad580afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 -- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fad5809b700 msgr2=0x7fad580b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fad5809b700 0x7fad580b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 -- 172.21.15.117:0/1212618398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580b0030 con 0x7fad58098e30 2020-11-20T13:56:06.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad577fe700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad58098e30 0x7fad58156fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fad4800c000 tx=0x7fad48006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad54ff9700 1 -- 172.21.15.117:0/1212618398 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad4800ca10 con 0x7fad58098e30 2020-11-20T13:56:06.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad54ff9700 1 -- 172.21.15.117:0/1212618398 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4800cb70 con 0x7fad58098e30 2020-11-20T13:56:06.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad54ff9700 1 -- 172.21.15.117:0/1212618398 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad480123f0 con 0x7fad58098e30 2020-11-20T13:56:06.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad580b0220 con 0x7fad58098e30 2020-11-20T13:56:06.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.591+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5814a9a0 con 0x7fad58098e30 2020-11-20T13:56:06.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.592+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad580af860 con 0x7fad58098e30 2020-11-20T13:56:06.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.592+0000 7fad54ff9700 1 -- 172.21.15.117:0/1212618398 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fad48012550 con 0x7fad58098e30 2020-11-20T13:56:06.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.593+0000 7fad54ff9700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fad38051210 0x7fad380536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:06.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.593+0000 7fad54ff9700 1 -- 172.21.15.117:0/1212618398 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fad48069400 con 0x7fad58098e30 2020-11-20T13:56:06.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.593+0000 7fad56ffd700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fad38051210 0x7fad380536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:06.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.593+0000 7fad56ffd700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fad38051210 0x7fad380536d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fad4c00d000 tx=0x7fad4c0079b0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:06.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.596+0000 7fad54ff9700 1 -- 172.21.15.117:0/1212618398 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fad480415c0 con 0x7fad58098e30 2020-11-20T13:56:06.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.796+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad581574d0 con 0x7fad58098e30 2020-11-20T13:56:06.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.797+0000 7fad54ff9700 1 -- 172.21.15.117:0/1212618398 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7fad48044be0 con 0x7fad58098e30 2020-11-20T13:56:06.799 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:06.799 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:55:51.676934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/1587825839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1587825839},{"type":"v1","addr":"172.21.15.117:6835","nonce":1587825839}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:06.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fad38051210 msgr2=0x7fad380536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fad38051210 0x7fad380536d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fad4c00d000 tx=0x7fad4c0079b0).stop 2020-11-20T13:56:06.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad58098e30 msgr2=0x7fad58156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:06.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad58098e30 0x7fad58156fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fad4800c000 tx=0x7fad48006dc0).stop 2020-11-20T13:56:06.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 shutdown_connections 2020-11-20T13:56:06.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fad38051210 0x7fad380536d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fad580ae6f0 0x7fad580afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad58098e30 0x7fad58156fe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 --2- 172.21.15.117:0/1212618398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fad5809b700 0x7fad580b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:06.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 >> 172.21.15.117:0/1212618398 conn(0x7fad5805b9f0 msgr2=0x7fad58141f90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:06.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.798+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 shutdown_connections 2020-11-20T13:56:06.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:06.799+0000 7fad5e246700 1 -- 172.21.15.117:0/1212618398 wait complete. 2020-11-20T13:56:06.803 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2020-11-20T13:56:07.010 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:55:50.675265+0000', 'modified': '2020-11-20T13:55:51.676934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/1587825839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1587825839}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1587825839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:07.011 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:07.011 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:07.138 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":4288,"want_state":"up:active","state":"up:active","rank_uptime":16.452984123,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":22.842912806000001} 2020-11-20T13:56:07.138 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 4288, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 16.452984123, "state": "up:active", "uptime": 22.842912806, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:07.139 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:07.139 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-11-20T13:56:07.139 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi117.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-11-20T13:56:07.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:07.140 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:07.156 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:07.223 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:07.224 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:07.289 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:07.290 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:07.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:07.291 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:07.344 INFO:teuthology.orchestra.run.smithi117.stdout:47 2020-11-20T13:56:07.344 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-11-20T13:56:07.345 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:07.386 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:07.413 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:07.414 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:07.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:2020-11-20T13:56:07.459+0000 7f3d6d1c51c0 -1 init, newargv = 0x55e35a9cace0 newargc=9 2020-11-20T13:56:07.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:ceph-fuse[26678]: starting ceph client 2020-11-20T13:56:07.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi117.stderr:ceph-fuse[26678]: starting fuse 2020-11-20T13:56:07.484 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:07.485 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:07.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:07.486 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:07.549 INFO:teuthology.orchestra.run.smithi117.stdout:47 2020-11-20T13:56:07.550 INFO:teuthology.orchestra.run.smithi117.stdout:48 2020-11-20T13:56:07.550 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-11-20T13:56:07.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:07.551 INFO:teuthology.orchestra.run.smithi117:> sudo python3 -c ' 2020-11-20T13:56:07.551 INFO:teuthology.orchestra.run.smithi117:> import glob 2020-11-20T13:56:07.552 INFO:teuthology.orchestra.run.smithi117:> import re 2020-11-20T13:56:07.552 INFO:teuthology.orchestra.run.smithi117:> import os 2020-11-20T13:56:07.552 INFO:teuthology.orchestra.run.smithi117:> import subprocess 2020-11-20T13:56:07.552 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:07.553 INFO:teuthology.orchestra.run.smithi117:> def find_socket(client_name): 2020-11-20T13:56:07.553 INFO:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-11-20T13:56:07.553 INFO:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2020-11-20T13:56:07.554 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:07.554 INFO:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2020-11-20T13:56:07.554 INFO:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2020-11-20T13:56:07.555 INFO:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2020-11-20T13:56:07.555 INFO:teuthology.orchestra.run.smithi117:> return files[0] 2020-11-20T13:56:07.555 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:07.555 INFO:teuthology.orchestra.run.smithi117:> for f in files: 2020-11-20T13:56:07.556 INFO:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:07.556 INFO:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:07.556 INFO:teuthology.orchestra.run.smithi117:> return f 2020-11-20T13:56:07.557 INFO:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:07.557 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:07.557 INFO:teuthology.orchestra.run.smithi117:> print(find_socket("client.1")) 2020-11-20T13:56:07.558 INFO:teuthology.orchestra.run.smithi117:> ' 2020-11-20T13:56:07.641 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.1.26678.asok 2020-11-20T13:56:07.646 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.26678.asok 2020-11-20T13:56:07.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:07.647 INFO:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.26678.asok status 2020-11-20T13:56:07.752 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2020-11-20T13:56:07.752 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2020-11-20T13:56:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "1", 2020-11-20T13:56:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2020-11-20T13:56:07.754 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2020-11-20T13:56:07.754 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "26678", 2020-11-20T13:56:07.754 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2020-11-20T13:56:07.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:07.754 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2020-11-20T13:56:07.755 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:07.755 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 4711, 2020-11-20T13:56:07.755 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2020-11-20T13:56:07.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2020-11-20T13:56:07.755 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2020-11-20T13:56:07.756 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 4711 2020-11-20T13:56:07.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:07.756 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:07.756 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:07.756 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:07.757 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 836851767 2020-11-20T13:56:07.757 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:07.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:07.762 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:07.762 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:07.762 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:07.762 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 836851767 2020-11-20T13:56:07.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:07.763 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.4711 172.21.15.117:0/836851767", 2020-11-20T13:56:07.763 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "172.21.15.117:0/836851767", 2020-11-20T13:56:07.763 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2020-11-20T13:56:07.763 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 5, 2020-11-20T13:56:07.763 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 25, 2020-11-20T13:56:07.764 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:07.764 INFO:teuthology.orchestra.run.smithi117.stdout: "blacklisted": false 2020-11-20T13:56:07.764 INFO:teuthology.orchestra.run.smithi117.stdout:} 2020-11-20T13:56:07.765 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:07.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.870+0000 7fba24569700 1 Processor -- start 2020-11-20T13:56:07.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.870+0000 7fba24569700 1 -- start start 2020-11-20T13:56:07.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba24569700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fba1c147190 0x7fba1c148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba24569700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c147880 con 0x7fba1c061f40 2020-11-20T13:56:07.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba24569700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c147b20 con 0x7fba1c09aea0 2020-11-20T13:56:07.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba24569700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c147dc0 con 0x7fba1c147190 2020-11-20T13:56:07.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba21b04700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fba1c09aea0 0x7fba1c0b8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:59972/0 (socket says 172.21.15.117:59972) 2020-11-20T13:56:07.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba21b04700 1 -- 172.21.15.117:0/647153744 learned_addr learned my addr 172.21.15.117:0/647153744 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:07.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba22305700 1 --2- 172.21.15.117:0/647153744 >> v2:172.21.15.85:3300/0 conn(0x7fba1c147190 0x7fba1c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.871+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291450129 0 0) 0x7fba1c147b20 con 0x7fba1c09aea0 2020-11-20T13:56:07.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba04003230 con 0x7fba1c09aea0 2020-11-20T13:56:07.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3150931549 0 0) 0x7fba04003230 con 0x7fba1c09aea0 2020-11-20T13:56:07.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 >> v2:172.21.15.85:3300/0 conn(0x7fba1c147190 msgr2=0x7fba1c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:07.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 --2- 172.21.15.117:0/647153744 >> v2:172.21.15.85:3300/0 conn(0x7fba1c147190 0x7fba1c148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 >> v1:172.21.15.85:6789/0 conn(0x7fba1c061f40 legacy=0x7fba1c14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c147570 con 0x7fba1c09aea0 2020-11-20T13:56:07.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fba180016d0 con 0x7fba1c09aea0 2020-11-20T13:56:07.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 >> v1:172.21.15.117:6789/0 conn(0x7fba1c09aea0 legacy=0x7fba1c0b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba04003720 0x7fba04003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba040044d0 0x7fba04004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba0400b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c147dc0 con 0x7fba04003720 2020-11-20T13:56:07.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04004340 con 0x7fba04004e40 2020-11-20T13:56:07.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0400bc40 con 0x7fba040044d0 2020-11-20T13:56:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba0400b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22305700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba04003720 0x7fba04003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21b04700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba040044d0 0x7fba04004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 -- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba040044d0 msgr2=0x7fba04004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:07.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba040044d0 0x7fba04004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 -- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba04003720 msgr2=0x7fba04003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:07.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba04003720 0x7fba04003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 -- 172.21.15.117:0/647153744 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04003230 con 0x7fba04004e40 2020-11-20T13:56:07.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 -- 172.21.15.117:0/647153744 reap_dead start 2020-11-20T13:56:07.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba22b06700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba0400b720 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fba1c0b1f20 tx=0x7fba14007e50).ready entity=mon.1 client_cookie=c6a16a0203eb06d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba1400ca10 con 0x7fba04004e40 2020-11-20T13:56:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1400cb70 con 0x7fba04004e40 2020-11-20T13:56:07.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba21303700 1 -- 172.21.15.117:0/647153744 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba14012400 con 0x7fba04004e40 2020-11-20T13:56:07.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba24569700 1 -- 172.21.15.117:0/647153744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 msgr2=0x7fba0400b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.872+0000 7fba24569700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba0400b720 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fba1c0b1f20 tx=0x7fba14007e50).stop 2020-11-20T13:56:07.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.873+0000 7fba24569700 1 -- 172.21.15.117:0/647153744 shutdown_connections 2020-11-20T13:56:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.873+0000 7fba24569700 1 --2- 172.21.15.117:0/647153744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba0400b720 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.873+0000 7fba24569700 1 -- 172.21.15.117:0/647153744 >> 172.21.15.117:0/647153744 conn(0x7fba1c05c4c0 msgr2=0x7fba1c0b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.873+0000 7fba24569700 1 -- 172.21.15.117:0/647153744 shutdown_connections 2020-11-20T13:56:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.873+0000 7fba24569700 1 -- 172.21.15.117:0/647153744 wait complete. 2020-11-20T13:56:07.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.874+0000 7fba24569700 1 Processor -- start 2020-11-20T13:56:07.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.874+0000 7fba24569700 1 -- start start 2020-11-20T13:56:07.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba1c1457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba1c0b8f80 0x7fba1c0649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba1c066eb0 0x7fba1c067330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c159810 con 0x7fba1c066eb0 2020-11-20T13:56:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0b7700 con 0x7fba04004e40 2020-11-20T13:56:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0b79a0 con 0x7fba1c0b8f80 2020-11-20T13:56:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba1c1457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba1c1457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40820/0 (socket says 172.21.15.117:40820) 2020-11-20T13:56:07.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 -- 172.21.15.117:0/2003996324 learned_addr learned my addr 172.21.15.117:0/2003996324 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:07.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22b06700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba1c066eb0 0x7fba1c067330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba21b04700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba1c0b8f80 0x7fba1c0649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 -- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba1c0b8f80 msgr2=0x7fba1c0649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba1c0b8f80 0x7fba1c0649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 -- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba1c066eb0 msgr2=0x7fba1c067330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:07.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba1c066eb0 0x7fba1c067330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:07.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 -- 172.21.15.117:0/2003996324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c067820 con 0x7fba04004e40 2020-11-20T13:56:07.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba22305700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba1c1457e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fba1c0b50c0 tx=0x7fba0c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:07.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba0b7fe700 1 -- 172.21.15.117:0/2003996324 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba0c00f910 con 0x7fba04004e40 2020-11-20T13:56:07.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba0b7fe700 1 -- 172.21.15.117:0/2003996324 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c00fa70 con 0x7fba04004e40 2020-11-20T13:56:07.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c0b89c0 con 0x7fba04004e40 2020-11-20T13:56:07.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba0b7fe700 1 -- 172.21.15.117:0/2003996324 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba0c00fdd0 con 0x7fba04004e40 2020-11-20T13:56:07.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.875+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c065050 con 0x7fba04004e40 2020-11-20T13:56:07.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.876+0000 7fba0b7fe700 1 -- 172.21.15.117:0/2003996324 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fba0c014440 con 0x7fba04004e40 2020-11-20T13:56:07.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.876+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c0b7b10 con 0x7fba04004e40 2020-11-20T13:56:07.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.876+0000 7fba0b7fe700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb9fc051290 0x7fb9fc053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:07.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.877+0000 7fba0b7fe700 1 -- 172.21.15.117:0/2003996324 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fba0c069830 con 0x7fba04004e40 2020-11-20T13:56:07.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.877+0000 7fba21b04700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb9fc051290 0x7fb9fc053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:07.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.877+0000 7fba21b04700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb9fc051290 0x7fb9fc053750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fba18005000 tx=0x7fba180076c0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:07.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:07.880+0000 7fba0b7fe700 1 -- 172.21.15.117:0/2003996324 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fba1c0b7b10 con 0x7fba04004e40 2020-11-20T13:56:08.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.030+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba1c065050 con 0x7fba04004e40 2020-11-20T13:56:08.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.031+0000 7fba0b7fe700 1 -- 172.21.15.117:0/2003996324 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fba1c065050 con 0x7fba04004e40 2020-11-20T13:56:08.032 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:08.033 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:08.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.032+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb9fc051290 msgr2=0x7fb9fc053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.032+0000 7fba24569700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb9fc051290 0x7fb9fc053750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fba18005000 tx=0x7fba180076c0).stop 2020-11-20T13:56:08.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.032+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 msgr2=0x7fba1c1457e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.032+0000 7fba24569700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba1c1457e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fba1c0b50c0 tx=0x7fba0c008040).stop 2020-11-20T13:56:08.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 shutdown_connections 2020-11-20T13:56:08.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb9fc051290 0x7fb9fc053750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fba1c066eb0 0x7fba1c067330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba04004e40 0x7fba1c1457e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 --2- 172.21.15.117:0/2003996324 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fba1c0b8f80 0x7fba1c0649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 >> 172.21.15.117:0/2003996324 conn(0x7fba1c05c4c0 msgr2=0x7fba1c0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 shutdown_connections 2020-11-20T13:56:08.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.033+0000 7fba24569700 1 -- 172.21.15.117:0/2003996324 wait complete. 2020-11-20T13:56:08.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:08.045 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:08.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.153+0000 7fe3f7fff700 1 Processor -- start 2020-11-20T13:56:08.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.153+0000 7fe3f7fff700 1 -- start start 2020-11-20T13:56:08.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f7fff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe3f805f300 0x7fe3f8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f809bf70 0x7fe3f8146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3f8062430 0x7fe3f8146e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f7fff700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f809ee60 con 0x7fe3f805f300 2020-11-20T13:56:08.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f7fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f809f100 con 0x7fe3f8062430 2020-11-20T13:56:08.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f7fff700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f809f3a0 con 0x7fe3f809bf70 2020-11-20T13:56:08.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3f8062430 0x7fe3f8146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3f8062430 0x7fe3f8146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40832/0 (socket says 172.21.15.117:40832) 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.154+0000 7fe3f77fe700 1 -- 172.21.15.117:0/4084493087 learned_addr learned my addr 172.21.15.117:0/4084493087 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f6ffd700 1 --2- 172.21.15.117:0/4084493087 >> v2:172.21.15.85:3300/0 conn(0x7fe3f805f300 0x7fe3f8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f67fc700 1 --2- 172.21.15.117:0/4084493087 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f809bf70 0x7fe3f8146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f77fe700 1 -- 172.21.15.117:0/4084493087 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f809bf70 msgr2=0x7fe3f8146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f77fe700 1 --2- 172.21.15.117:0/4084493087 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f809bf70 0x7fe3f8146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f77fe700 1 -- 172.21.15.117:0/4084493087 >> v2:172.21.15.85:3300/0 conn(0x7fe3f805f300 msgr2=0x7fe3f8062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f77fe700 1 --2- 172.21.15.117:0/4084493087 >> v2:172.21.15.85:3300/0 conn(0x7fe3f805f300 0x7fe3f8062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f77fe700 1 -- 172.21.15.117:0/4084493087 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f805f6e0 con 0x7fe3f8062430 2020-11-20T13:56:08.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f77fe700 1 --2- 172.21.15.117:0/4084493087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f8062430 0x7fe3f8146e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fe3f00032b0 tx=0x7fe3f0009270).ready entity=mon.1 client_cookie=47837073ee8b2d7c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f5ffb700 1 -- 172.21.15.117:0/4084493087 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3f000bc00 con 0x7fe3f8062430 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f5ffb700 1 -- 172.21.15.117:0/4084493087 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000bd60 con 0x7fe3f8062430 2020-11-20T13:56:08.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f5ffb700 1 -- 172.21.15.117:0/4084493087 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3f0011400 con 0x7fe3f8062430 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f7fff700 1 -- 172.21.15.117:0/4084493087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f8062430 msgr2=0x7fe3f8146e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.155+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/4084493087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f8062430 0x7fe3f8146e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fe3f00032b0 tx=0x7fe3f0009270).stop 2020-11-20T13:56:08.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.156+0000 7fe3f7fff700 1 -- 172.21.15.117:0/4084493087 shutdown_connections 2020-11-20T13:56:08.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.156+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/4084493087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f8062430 0x7fe3f8146e10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.156+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/4084493087 >> v2:172.21.15.85:3300/0 conn(0x7fe3f805f300 0x7fe3f8062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.156+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/4084493087 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f809bf70 0x7fe3f8146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.156+0000 7fe3f7fff700 1 -- 172.21.15.117:0/4084493087 >> 172.21.15.117:0/4084493087 conn(0x7fe3f80b9490 msgr2=0x7fe3f80b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.156+0000 7fe3f7fff700 1 -- 172.21.15.117:0/4084493087 shutdown_connections 2020-11-20T13:56:08.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.156+0000 7fe3f7fff700 1 -- 172.21.15.117:0/4084493087 wait complete. 2020-11-20T13:56:08.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 Processor -- start 2020-11-20T13:56:08.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 -- start start 2020-11-20T13:56:08.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f805f300 0x7fe3f8145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe3f8062430 0x7fe3f8153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f809bf70 0x7fe3f80b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80b1d10 con 0x7fe3f8062430 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80b3700 con 0x7fe3f809bf70 2020-11-20T13:56:08.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f7fff700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80b39a0 con 0x7fe3f805f300 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f809bf70 0x7fe3f80b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f809bf70 0x7fe3f80b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40838/0 (socket says 172.21.15.117:40838) 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f77fe700 1 -- 172.21.15.117:0/3734914130 learned_addr learned my addr 172.21.15.117:0/3734914130 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f67fc700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe3f8062430 0x7fe3f8153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.157+0000 7fe3f6ffd700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f805f300 0x7fe3f8145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f77fe700 1 -- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f805f300 msgr2=0x7fe3f8145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f77fe700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f805f300 0x7fe3f8145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f77fe700 1 -- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe3f8062430 msgr2=0x7fe3f8153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f77fe700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe3f8062430 0x7fe3f8153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f77fe700 1 -- 172.21.15.117:0/3734914130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80b3c40 con 0x7fe3f809bf70 2020-11-20T13:56:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f77fe700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f809bf70 0x7fe3f80b2ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe3f8098940 tx=0x7fe3f0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3e7fff700 1 -- 172.21.15.117:0/3734914130 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3f001f670 con 0x7fe3f809bf70 2020-11-20T13:56:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3e7fff700 1 -- 172.21.15.117:0/3734914130 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f001f7d0 con 0x7fe3f809bf70 2020-11-20T13:56:08.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f80b40e0 con 0x7fe3f809bf70 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3e7fff700 1 -- 172.21.15.117:0/3734914130 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3f001fb30 con 0x7fe3f809bf70 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.158+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8099420 con 0x7fe3f809bf70 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.159+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f80b3570 con 0x7fe3f809bf70 2020-11-20T13:56:08.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.159+0000 7fe3e7fff700 1 -- 172.21.15.117:0/3734914130 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe3f0026030 con 0x7fe3f809bf70 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.159+0000 7fe3e7fff700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe3d4051290 0x7fe3d4053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.159+0000 7fe3e7fff700 1 -- 172.21.15.117:0/3734914130 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe3f006a520 con 0x7fe3f809bf70 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.159+0000 7fe3f6ffd700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe3d4051290 0x7fe3d4053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.159+0000 7fe3f6ffd700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe3d4051290 0x7fe3d4053750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fe3ec00c000 tx=0x7fe3ec008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.162+0000 7fe3e7fff700 1 -- 172.21.15.117:0/3734914130 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe3f0042740 con 0x7fe3f809bf70 2020-11-20T13:56:08.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.312+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe3f80999f0 con 0x7fe3f809bf70 2020-11-20T13:56:08.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.312+0000 7fe3e7fff700 1 -- 172.21.15.117:0/3734914130 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe3f00150b0 con 0x7fe3f809bf70 2020-11-20T13:56:08.314 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:08.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe3d4051290 msgr2=0x7fe3d4053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe3d4051290 0x7fe3d4053750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fe3ec00c000 tx=0x7fe3ec008040).stop 2020-11-20T13:56:08.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f809bf70 msgr2=0x7fe3f80b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f809bf70 0x7fe3f80b2ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe3f8098940 tx=0x7fe3f0016000).stop 2020-11-20T13:56:08.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 shutdown_connections 2020-11-20T13:56:08.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe3d4051290 0x7fe3d4053750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe3f805f300 0x7fe3f8145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3f809bf70 0x7fe3f80b2ff0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 --2- 172.21.15.117:0/3734914130 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe3f8062430 0x7fe3f8153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 >> 172.21.15.117:0/3734914130 conn(0x7fe3f80b9490 msgr2=0x7fe3f81466f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 shutdown_connections 2020-11-20T13:56:08.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.314+0000 7fe3f7fff700 1 -- 172.21.15.117:0/3734914130 wait complete. 2020-11-20T13:56:08.325 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:08.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.434+0000 7ff21a21e700 1 Processor -- start 2020-11-20T13:56:08.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.434+0000 7ff21a21e700 1 -- start start 2020-11-20T13:56:08.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff21a21e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7ff21409cdd0 0x7ff214147c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff21a21e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff21405ee90 0x7ff214067200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff21a21e700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff214066890 con 0x7ff214146840 2020-11-20T13:56:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff21a21e700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff214148140 con 0x7ff21409cdd0 2020-11-20T13:56:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff21a21e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff214066ac0 con 0x7ff21405ee90 2020-11-20T13:56:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff218a1b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff21405ee90 0x7ff214067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff218a1b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff21405ee90 0x7ff214067200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40842/0 (socket says 172.21.15.117:40842) 2020-11-20T13:56:08.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff218a1b700 1 -- 172.21.15.117:0/3959295677 learned_addr learned my addr 172.21.15.117:0/3959295677 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:08.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.435+0000 7ff21921c700 1 --2- 172.21.15.117:0/3959295677 >> v2:172.21.15.85:3300/0 conn(0x7ff21409cdd0 0x7ff214147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff218a1b700 1 -- 172.21.15.117:0/3959295677 >> v2:172.21.15.85:3300/0 conn(0x7ff21409cdd0 msgr2=0x7ff214147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff218a1b700 1 --2- 172.21.15.117:0/3959295677 >> v2:172.21.15.85:3300/0 conn(0x7ff21409cdd0 0x7ff214147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff218a1b700 1 -- 172.21.15.117:0/3959295677 >> v1:172.21.15.85:6789/0 conn(0x7ff214146840 legacy=0x7ff21409e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff218a1b700 1 -- 172.21.15.117:0/3959295677 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214066d60 con 0x7ff21405ee90 2020-11-20T13:56:08.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff218a1b700 1 --2- 172.21.15.117:0/3959295677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff21405ee90 0x7ff214067200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff208002f30 tx=0x7ff208006e80).ready entity=mon.1 client_cookie=a015027a0659d6a9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff207fff700 1 -- 172.21.15.117:0/3959295677 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2080097f0 con 0x7ff21405ee90 2020-11-20T13:56:08.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff207fff700 1 -- 172.21.15.117:0/3959295677 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff208009950 con 0x7ff21405ee90 2020-11-20T13:56:08.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff207fff700 1 -- 172.21.15.117:0/3959295677 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff208009cb0 con 0x7ff21405ee90 2020-11-20T13:56:08.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 -- 172.21.15.117:0/3959295677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff21405ee90 msgr2=0x7ff214067200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 --2- 172.21.15.117:0/3959295677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff21405ee90 0x7ff214067200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff208002f30 tx=0x7ff208006e80).stop 2020-11-20T13:56:08.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 -- 172.21.15.117:0/3959295677 shutdown_connections 2020-11-20T13:56:08.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 --2- 172.21.15.117:0/3959295677 >> v2:172.21.15.85:3300/0 conn(0x7ff21409cdd0 0x7ff214147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 --2- 172.21.15.117:0/3959295677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff21405ee90 0x7ff214067200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.442 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 -- 172.21.15.117:0/3959295677 >> 172.21.15.117:0/3959295677 conn(0x7ff2140bd890 msgr2=0x7ff2140bdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 -- 172.21.15.117:0/3959295677 shutdown_connections 2020-11-20T13:56:08.443 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.436+0000 7ff21a21e700 1 -- 172.21.15.117:0/3959295677 wait complete. 2020-11-20T13:56:08.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.437+0000 7ff21a21e700 1 Processor -- start 2020-11-20T13:56:08.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.437+0000 7ff21a21e700 1 -- start start 2020-11-20T13:56:08.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff21a21e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff21405ee90 0x7ff214156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff21a21e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff21409cdd0 0x7ff2141457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff21a21e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214146840 0x7ff2140a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff21a21e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214159770 con 0x7ff21409cdd0 2020-11-20T13:56:08.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff21a21e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214159a10 con 0x7ff214146840 2020-11-20T13:56:08.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff21a21e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff214159cb0 con 0x7ff21405ee90 2020-11-20T13:56:08.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214146840 0x7ff2140a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214146840 0x7ff2140a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40850/0 (socket says 172.21.15.117:40850) 2020-11-20T13:56:08.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 -- 172.21.15.117:0/1581902681 learned_addr learned my addr 172.21.15.117:0/1581902681 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:08.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff21921c700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff21405ee90 0x7ff214156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff218a1b700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff21409cdd0 0x7ff2141457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 -- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff21405ee90 msgr2=0x7ff214156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff21405ee90 0x7ff214156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 -- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff21409cdd0 msgr2=0x7ff2141457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff21409cdd0 0x7ff2141457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.438+0000 7ff219a1d700 1 -- 172.21.15.117:0/1581902681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214159f50 con 0x7ff214146840 2020-11-20T13:56:08.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.439+0000 7ff219a1d700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214146840 0x7ff2140a1af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7ff214060990 tx=0x7ff210007b00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.439+0000 7ff2067fc700 1 -- 172.21.15.117:0/1581902681 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff21000e4a0 con 0x7ff214146840 2020-11-20T13:56:08.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.439+0000 7ff2067fc700 1 -- 172.21.15.117:0/1581902681 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000e600 con 0x7ff214146840 2020-11-20T13:56:08.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.439+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2140a2970 con 0x7ff214146840 2020-11-20T13:56:08.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.439+0000 7ff2067fc700 1 -- 172.21.15.117:0/1581902681 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff21000e960 con 0x7ff214146840 2020-11-20T13:56:08.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.439+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2140a1fe0 con 0x7ff214146840 2020-11-20T13:56:08.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.440+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff214057050 con 0x7ff214146840 2020-11-20T13:56:08.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.440+0000 7ff2067fc700 1 -- 172.21.15.117:0/1581902681 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff210010030 con 0x7ff214146840 2020-11-20T13:56:08.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.440+0000 7ff2067fc700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff1f40513c0 0x7ff1f4053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.440+0000 7ff2067fc700 1 -- 172.21.15.117:0/1581902681 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff210068f20 con 0x7ff214146840 2020-11-20T13:56:08.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.440+0000 7ff21921c700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff1f40513c0 0x7ff1f4053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.441+0000 7ff21921c700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff1f40513c0 0x7ff1f4053880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff20c00c000 tx=0x7ff20c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.444+0000 7ff2067fc700 1 -- 172.21.15.117:0/1581902681 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff2100410e0 con 0x7ff214146840 2020-11-20T13:56:08.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.592+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff21405ca00 con 0x7ff214146840 2020-11-20T13:56:08.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.593+0000 7ff2067fc700 1 -- 172.21.15.117:0/1581902681 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff210044700 con 0x7ff214146840 2020-11-20T13:56:08.594 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff1f40513c0 msgr2=0x7ff1f4053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff1f40513c0 0x7ff1f4053880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7ff20c00c000 tx=0x7ff20c008040).stop 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214146840 msgr2=0x7ff2140a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214146840 0x7ff2140a1af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7ff214060990 tx=0x7ff210007b00).stop 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 shutdown_connections 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff1f40513c0 0x7ff1f4053880 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff21405ee90 0x7ff214156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff214146840 0x7ff2140a1af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 --2- 172.21.15.117:0/1581902681 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff21409cdd0 0x7ff2141457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 >> 172.21.15.117:0/1581902681 conn(0x7ff2140bd890 msgr2=0x7ff2141447c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 shutdown_connections 2020-11-20T13:56:08.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.594+0000 7ff21a21e700 1 -- 172.21.15.117:0/1581902681 wait complete. 2020-11-20T13:56:08.605 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.713+0000 7fd3e80e5700 1 Processor -- start 2020-11-20T13:56:08.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.714+0000 7fd3e80e5700 1 -- start start 2020-11-20T13:56:08.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.714+0000 7fd3e80e5700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3e009bd80 0x7fd3e009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.714+0000 7fd3e80e5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e009a110 0x7fd3e01437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.714+0000 7fd3e80e5700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e00999d0 con 0x7fd3e0143c90 2020-11-20T13:56:08.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.714+0000 7fd3e80e5700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e0099c70 con 0x7fd3e009bd80 2020-11-20T13:56:08.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.714+0000 7fd3e80e5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0144120 con 0x7fd3e009a110 2020-11-20T13:56:08.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.714+0000 7fd3e5e81700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3e009bd80 0x7fd3e009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5e81700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3e009bd80 0x7fd3e009cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40854/0 (socket says 172.21.15.117:40854) 2020-11-20T13:56:08.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5e81700 1 -- 172.21.15.117:0/2518894026 learned_addr learned my addr 172.21.15.117:0/2518894026 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:08.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5680700 1 --2- 172.21.15.117:0/2518894026 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e009a110 0x7fd3e01437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5e81700 1 -- 172.21.15.117:0/2518894026 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e009a110 msgr2=0x7fd3e01437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5e81700 1 --2- 172.21.15.117:0/2518894026 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e009a110 0x7fd3e01437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5e81700 1 -- 172.21.15.117:0/2518894026 >> v1:172.21.15.85:6789/0 conn(0x7fd3e0143c90 legacy=0x7fd3e00b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5e81700 1 -- 172.21.15.117:0/2518894026 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e009c160 con 0x7fd3e009bd80 2020-11-20T13:56:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e5e81700 1 --2- 172.21.15.117:0/2518894026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e009cb40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fd3e009b9c0 tx=0x7fd3d0006e80).ready entity=mon.1 client_cookie=e4f8f37a14918c45 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e4e7f700 1 -- 172.21.15.117:0/2518894026 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3d00097b0 con 0x7fd3e009bd80 2020-11-20T13:56:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e4e7f700 1 -- 172.21.15.117:0/2518894026 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d0009910 con 0x7fd3e009bd80 2020-11-20T13:56:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e4e7f700 1 -- 172.21.15.117:0/2518894026 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3d0009c70 con 0x7fd3e009bd80 2020-11-20T13:56:08.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e80e5700 1 -- 172.21.15.117:0/2518894026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 msgr2=0x7fd3e009cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/2518894026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e009cb40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fd3e009b9c0 tx=0x7fd3d0006e80).stop 2020-11-20T13:56:08.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e80e5700 1 -- 172.21.15.117:0/2518894026 shutdown_connections 2020-11-20T13:56:08.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/2518894026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e009cb40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/2518894026 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e009a110 0x7fd3e01437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.715+0000 7fd3e80e5700 1 -- 172.21.15.117:0/2518894026 >> 172.21.15.117:0/2518894026 conn(0x7fd3e00b9770 msgr2=0x7fd3e00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.722 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.716+0000 7fd3e80e5700 1 -- 172.21.15.117:0/2518894026 shutdown_connections 2020-11-20T13:56:08.722 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.716+0000 7fd3e80e5700 1 -- 172.21.15.117:0/2518894026 wait complete. 2020-11-20T13:56:08.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.716+0000 7fd3e80e5700 1 Processor -- start 2020-11-20T13:56:08.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e80e5700 1 -- start start 2020-11-20T13:56:08.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e80e5700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3e009a110 0x7fd3e0067470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e80e5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e0144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e80e5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e0143c90 0x7fd3e0144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e80e5700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0157e60 con 0x7fd3e009a110 2020-11-20T13:56:08.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e80e5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0158100 con 0x7fd3e009bd80 2020-11-20T13:56:08.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e80e5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e01583a0 con 0x7fd3e0143c90 2020-11-20T13:56:08.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e5680700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e0144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e5680700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e0144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40862/0 (socket says 172.21.15.117:40862) 2020-11-20T13:56:08.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e5680700 1 -- 172.21.15.117:0/1244818447 learned_addr learned my addr 172.21.15.117:0/1244818447 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:08.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e6682700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e0143c90 0x7fd3e0144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.717+0000 7fd3e5e81700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3e009a110 0x7fd3e0067470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e5680700 1 -- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e0143c90 msgr2=0x7fd3e0144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.728 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e5680700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e0143c90 0x7fd3e0144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e5680700 1 -- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3e009a110 msgr2=0x7fd3e0067470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e5680700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3e009a110 0x7fd3e0067470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e5680700 1 -- 172.21.15.117:0/1244818447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0157ae0 con 0x7fd3e009bd80 2020-11-20T13:56:08.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e5680700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e0144260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd3dc00c000 tx=0x7fd3dc006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3ceffd700 1 -- 172.21.15.117:0/1244818447 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc00ca10 con 0x7fd3e009bd80 2020-11-20T13:56:08.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3ceffd700 1 -- 172.21.15.117:0/1244818447 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc00cb70 con 0x7fd3e009bd80 2020-11-20T13:56:08.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e0144c40 con 0x7fd3e009bd80 2020-11-20T13:56:08.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3ceffd700 1 -- 172.21.15.117:0/1244818447 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc0123f0 con 0x7fd3e009bd80 2020-11-20T13:56:08.730 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.718+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e00b1b50 con 0x7fd3e009bd80 2020-11-20T13:56:08.731 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.719+0000 7fd3ceffd700 1 -- 172.21.15.117:0/1244818447 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd3dc012550 con 0x7fd3e009bd80 2020-11-20T13:56:08.731 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.719+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e0158560 con 0x7fd3e009bd80 2020-11-20T13:56:08.731 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.719+0000 7fd3ceffd700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3c0051320 0x7fd3c00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.731 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.719+0000 7fd3ceffd700 1 -- 172.21.15.117:0/1244818447 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd3dc069510 con 0x7fd3e009bd80 2020-11-20T13:56:08.731 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.719+0000 7fd3e5e81700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3c0051320 0x7fd3c00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.732 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.719+0000 7fd3e5e81700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3c0051320 0x7fd3c00537e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd3d001c000 tx=0x7fd3d0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.732 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.722+0000 7fd3ceffd700 1 -- 172.21.15.117:0/1244818447 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd3dc0416d0 con 0x7fd3e009bd80 2020-11-20T13:56:08.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.870+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3e0057050 con 0x7fd3e009bd80 2020-11-20T13:56:08.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.871+0000 7fd3ceffd700 1 -- 172.21.15.117:0/1244818447 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd3dc044cf0 con 0x7fd3e009bd80 2020-11-20T13:56:08.872 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:08.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3c0051320 msgr2=0x7fd3c00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3c0051320 0x7fd3c00537e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd3d001c000 tx=0x7fd3d0015000).stop 2020-11-20T13:56:08.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 msgr2=0x7fd3e0144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e0144260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd3dc00c000 tx=0x7fd3dc006dc0).stop 2020-11-20T13:56:08.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 shutdown_connections 2020-11-20T13:56:08.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3c0051320 0x7fd3c00537e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3e0143c90 0x7fd3e0144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3e009a110 0x7fd3e0067470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 --2- 172.21.15.117:0/1244818447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3e009bd80 0x7fd3e0144260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 >> 172.21.15.117:0/1244818447 conn(0x7fd3e00b9770 msgr2=0x7fd3e0142b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:08.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 shutdown_connections 2020-11-20T13:56:08.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.872+0000 7fd3e80e5700 1 -- 172.21.15.117:0/1244818447 wait complete. 2020-11-20T13:56:08.883 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:08.993 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8722ea9700 1 Processor -- start 2020-11-20T13:56:08.993 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8722ea9700 1 -- start start 2020-11-20T13:56:08.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8722ea9700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f871c09bd80 0x7f871c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8722ea9700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f871c143eb0 0x7f871c0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:08.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8722ea9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c0999d0 con 0x7f871c09a110 2020-11-20T13:56:08.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8722ea9700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c099c70 con 0x7f871c09bd80 2020-11-20T13:56:08.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8722ea9700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c09c160 con 0x7f871c143eb0 2020-11-20T13:56:08.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.992+0000 7f8721446700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f871c143eb0 0x7f871c0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f871bfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f871c09a110 0x7f871c1437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60030/0 (socket says 172.21.15.117:60030) 2020-11-20T13:56:08.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8721446700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f871c143eb0 0x7f871c0b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40872/0 (socket says 172.21.15.117:40872) 2020-11-20T13:56:08.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f871bfff700 1 -- 172.21.15.117:0/1427685362 learned_addr learned my addr 172.21.15.117:0/1427685362 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:08.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8720c45700 1 --2- 172.21.15.117:0/1427685362 >> v2:172.21.15.85:3300/0 conn(0x7f871c09bd80 0x7f871c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:08.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8721446700 1 -- 172.21.15.117:0/1427685362 >> v2:172.21.15.85:3300/0 conn(0x7f871c09bd80 msgr2=0x7f871c09cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:08.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8721446700 1 --2- 172.21.15.117:0/1427685362 >> v2:172.21.15.85:3300/0 conn(0x7f871c09bd80 0x7f871c09cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8721446700 1 -- 172.21.15.117:0/1427685362 >> v1:172.21.15.117:6789/0 conn(0x7f871c09a110 legacy=0x7f871c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8721446700 1 -- 172.21.15.117:0/1427685362 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c09a5c0 con 0x7f871c143eb0 2020-11-20T13:56:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8721446700 1 --2- 172.21.15.117:0/1427685362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c143eb0 0x7f871c0b4110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f870c0010d0 tx=0x7f87140091a0).ready entity=mon.1 client_cookie=ca8e96278410fe2e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f871b7fe700 1 -- 172.21.15.117:0/1427685362 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f871400b860 con 0x7f871c143eb0 2020-11-20T13:56:08.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f871b7fe700 1 -- 172.21.15.117:0/1427685362 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871400b9c0 con 0x7f871c143eb0 2020-11-20T13:56:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f871b7fe700 1 -- 172.21.15.117:0/1427685362 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f871400bb30 con 0x7f871c143eb0 2020-11-20T13:56:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8722ea9700 1 -- 172.21.15.117:0/1427685362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c143eb0 msgr2=0x7f871c0b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.993+0000 7f8722ea9700 1 --2- 172.21.15.117:0/1427685362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c143eb0 0x7f871c0b4110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f870c0010d0 tx=0x7f87140091a0).stop 2020-11-20T13:56:09.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.994+0000 7f8722ea9700 1 -- 172.21.15.117:0/1427685362 shutdown_connections 2020-11-20T13:56:09.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.994+0000 7f8722ea9700 1 --2- 172.21.15.117:0/1427685362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c143eb0 0x7f871c0b4110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.994+0000 7f8722ea9700 1 --2- 172.21.15.117:0/1427685362 >> v2:172.21.15.85:3300/0 conn(0x7f871c09bd80 0x7f871c09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.994+0000 7f8722ea9700 1 -- 172.21.15.117:0/1427685362 >> 172.21.15.117:0/1427685362 conn(0x7f871c0b9770 msgr2=0x7f871c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.994+0000 7f8722ea9700 1 -- 172.21.15.117:0/1427685362 shutdown_connections 2020-11-20T13:56:09.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.994+0000 7f8722ea9700 1 -- 172.21.15.117:0/1427685362 wait complete. 2020-11-20T13:56:09.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 Processor -- start 2020-11-20T13:56:09.002 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 -- start start 2020-11-20T13:56:09.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f871c09a110 0x7f871c066e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c09bd80 0x7f871c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f871c143eb0 0x7f871c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c154980 con 0x7f871c143eb0 2020-11-20T13:56:09.005 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c154b70 con 0x7f871c09bd80 2020-11-20T13:56:09.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.995+0000 7f8722ea9700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c154e10 con 0x7f871c09a110 2020-11-20T13:56:09.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c09bd80 0x7f871c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c09bd80 0x7f871c154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40876/0 (socket says 172.21.15.117:40876) 2020-11-20T13:56:09.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 -- 172.21.15.117:0/3083640503 learned_addr learned my addr 172.21.15.117:0/3083640503 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:09.006 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f8720c45700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f871c09a110 0x7f871c066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.007 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f8721446700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f871c143eb0 0x7f871c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.007 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 -- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f871c09a110 msgr2=0x7f871c066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f871c09a110 0x7f871c066e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 -- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f871c143eb0 msgr2=0x7f871c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f871c143eb0 0x7f871c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 -- 172.21.15.117:0/3083640503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c1550b0 con 0x7f871c09bd80 2020-11-20T13:56:09.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f871bfff700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c09bd80 0x7f871c154490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f870c001190 tx=0x7f870c007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.009 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f8719ffb700 1 -- 172.21.15.117:0/3083640503 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870c007320 con 0x7f871c09bd80 2020-11-20T13:56:09.009 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f8719ffb700 1 -- 172.21.15.117:0/3083640503 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c007480 con 0x7f871c09bd80 2020-11-20T13:56:09.009 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c148f00 con 0x7f871c09bd80 2020-11-20T13:56:09.009 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f8719ffb700 1 -- 172.21.15.117:0/3083640503 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870c0057d0 con 0x7f871c09bd80 2020-11-20T13:56:09.009 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.996+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c158540 con 0x7f871c09bd80 2020-11-20T13:56:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.997+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c155240 con 0x7f871c09bd80 2020-11-20T13:56:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.997+0000 7f8719ffb700 1 -- 172.21.15.117:0/3083640503 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f870c010030 con 0x7f871c09bd80 2020-11-20T13:56:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.998+0000 7f8719ffb700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8700051320 0x7f87000537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.998+0000 7f8719ffb700 1 -- 172.21.15.117:0/3083640503 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f870c068f70 con 0x7f871c09bd80 2020-11-20T13:56:09.010 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.998+0000 7f8720c45700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8700051320 0x7f87000537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.011 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:08.998+0000 7f8720c45700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8700051320 0x7f87000537e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f870400b000 tx=0x7f8704006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.011 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.001+0000 7f8719ffb700 1 -- 172.21.15.117:0/3083640503 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f870c041190 con 0x7f871c09bd80 2020-11-20T13:56:09.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.151+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f871c057050 con 0x7f871c09bd80 2020-11-20T13:56:09.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.151+0000 7f8719ffb700 1 -- 172.21.15.117:0/3083640503 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f870c0447b0 con 0x7f871c09bd80 2020-11-20T13:56:09.153 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:09.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8700051320 msgr2=0x7f87000537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8700051320 0x7f87000537e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f870400b000 tx=0x7f8704006dc0).stop 2020-11-20T13:56:09.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c09bd80 msgr2=0x7f871c154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c09bd80 0x7f871c154490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f870c001190 tx=0x7f870c007b50).stop 2020-11-20T13:56:09.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 shutdown_connections 2020-11-20T13:56:09.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8700051320 0x7f87000537e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f871c09a110 0x7f871c066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f871c143eb0 0x7f871c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 --2- 172.21.15.117:0/3083640503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f871c09bd80 0x7f871c154490 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 >> 172.21.15.117:0/3083640503 conn(0x7f871c0b9770 msgr2=0x7f871c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.152+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 shutdown_connections 2020-11-20T13:56:09.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.153+0000 7f8722ea9700 1 -- 172.21.15.117:0/3083640503 wait complete. 2020-11-20T13:56:09.164 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:09.164 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:09.272 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.270+0000 7f354be62700 1 Processor -- start 2020-11-20T13:56:09.272 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354be62700 1 -- start start 2020-11-20T13:56:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354be62700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f354405cd80 0x7f3544147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354be62700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f354414a830 0x7f3544140d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354be62700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35440b6780 con 0x7f3544141260 2020-11-20T13:56:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354be62700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3544147850 con 0x7f354405cd80 2020-11-20T13:56:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354be62700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3544147af0 con 0x7f354414a830 2020-11-20T13:56:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f35493fd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f354414a830 0x7f3544140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.275 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354a3ff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3544141260 0x7f3544060630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60046/0 (socket says 172.21.15.117:60046) 2020-11-20T13:56:09.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.271+0000 7f354a3ff700 1 -- 172.21.15.117:0/881663132 learned_addr learned my addr 172.21.15.117:0/881663132 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:09.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f3549bfe700 1 --2- 172.21.15.117:0/881663132 >> v2:172.21.15.85:3300/0 conn(0x7f354405cd80 0x7f3544147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f3548bfc700 1 -- 172.21.15.117:0/881663132 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848404898 0 0) 0x7f35440b6780 con 0x7f3544141260 2020-11-20T13:56:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f3548bfc700 1 -- 172.21.15.117:0/881663132 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f352c003230 con 0x7f3544141260 2020-11-20T13:56:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f35493fd700 1 -- 172.21.15.117:0/881663132 >> v2:172.21.15.85:3300/0 conn(0x7f354405cd80 msgr2=0x7f3544147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f35493fd700 1 --2- 172.21.15.117:0/881663132 >> v2:172.21.15.85:3300/0 conn(0x7f354405cd80 0x7f3544147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f35493fd700 1 -- 172.21.15.117:0/881663132 >> v1:172.21.15.117:6789/0 conn(0x7f3544141260 legacy=0x7f3544060630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f35493fd700 1 -- 172.21.15.117:0/881663132 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544147d90 con 0x7f354414a830 2020-11-20T13:56:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f35493fd700 1 --2- 172.21.15.117:0/881663132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354414a830 0x7f3544140d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f3534002f30 tx=0x7f3534006dc0).ready entity=mon.1 client_cookie=c0627aa5813d7a2b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f3548bfc700 1 -- 172.21.15.117:0/881663132 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35340096f0 con 0x7f354414a830 2020-11-20T13:56:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f3548bfc700 1 -- 172.21.15.117:0/881663132 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3534009850 con 0x7f354414a830 2020-11-20T13:56:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f3548bfc700 1 -- 172.21.15.117:0/881663132 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3534009bb0 con 0x7f354414a830 2020-11-20T13:56:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f354be62700 1 -- 172.21.15.117:0/881663132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354414a830 msgr2=0x7f3544140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.272+0000 7f354be62700 1 --2- 172.21.15.117:0/881663132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354414a830 0x7f3544140d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f3534002f30 tx=0x7f3534006dc0).stop 2020-11-20T13:56:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.273+0000 7f354be62700 1 -- 172.21.15.117:0/881663132 shutdown_connections 2020-11-20T13:56:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.273+0000 7f354be62700 1 --2- 172.21.15.117:0/881663132 >> v2:172.21.15.85:3300/0 conn(0x7f354405cd80 0x7f3544147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.273+0000 7f354be62700 1 --2- 172.21.15.117:0/881663132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354414a830 0x7f3544140d70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.280 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.273+0000 7f354be62700 1 -- 172.21.15.117:0/881663132 >> 172.21.15.117:0/881663132 conn(0x7f35440b7480 msgr2=0x7f3544062f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.280 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.273+0000 7f354be62700 1 -- 172.21.15.117:0/881663132 shutdown_connections 2020-11-20T13:56:09.280 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.273+0000 7f354be62700 1 -- 172.21.15.117:0/881663132 wait complete. 2020-11-20T13:56:09.282 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 Processor -- start 2020-11-20T13:56:09.282 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 -- start start 2020-11-20T13:56:09.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f354405cd80 0x7f3544146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3544141260 0x7f354409ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f354414a830 0x7f354415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.283 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354409e7e0 con 0x7f354405cd80 2020-11-20T13:56:09.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354415ac30 con 0x7f3544141260 2020-11-20T13:56:09.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f354be62700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354409f6b0 con 0x7f354414a830 2020-11-20T13:56:09.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f35493fd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3544141260 0x7f354409ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f35493fd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3544141260 0x7f354409ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40890/0 (socket says 172.21.15.117:40890) 2020-11-20T13:56:09.284 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.274+0000 7f35493fd700 1 -- 172.21.15.117:0/2982232398 learned_addr learned my addr 172.21.15.117:0/2982232398 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:09.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f354a3ff700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f354414a830 0x7f354415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f3549bfe700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f354405cd80 0x7f3544146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f35493fd700 1 -- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f354414a830 msgr2=0x7f354415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f35493fd700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f354414a830 0x7f354415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.285 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f35493fd700 1 -- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f354405cd80 msgr2=0x7f3544146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f35493fd700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f354405cd80 0x7f3544146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f35493fd700 1 -- 172.21.15.117:0/2982232398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354409f950 con 0x7f3544141260 2020-11-20T13:56:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f35493fd700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3544141260 0x7f354409ec50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f3534002f30 tx=0x7f3534015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f3532ffd700 1 -- 172.21.15.117:0/2982232398 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3534009e30 con 0x7f3544141260 2020-11-20T13:56:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f3532ffd700 1 -- 172.21.15.117:0/2982232398 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3534011bb0 con 0x7f3544141260 2020-11-20T13:56:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354409fbf0 con 0x7f3544141260 2020-11-20T13:56:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f3532ffd700 1 -- 172.21.15.117:0/2982232398 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f353401d450 con 0x7f3544141260 2020-11-20T13:56:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.275+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3544159ce0 con 0x7f3544141260 2020-11-20T13:56:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.276+0000 7f3532ffd700 1 -- 172.21.15.117:0/2982232398 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f353400f070 con 0x7f3544141260 2020-11-20T13:56:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.276+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354409e950 con 0x7f3544141260 2020-11-20T13:56:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.276+0000 7f3532ffd700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3524051210 0x7f35240536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.277+0000 7f3532ffd700 1 -- 172.21.15.117:0/2982232398 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f353406aa30 con 0x7f3544141260 2020-11-20T13:56:09.289 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.277+0000 7f3549bfe700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3524051210 0x7f35240536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:09.289 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.277+0000 7f3549bfe700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3524051210 0x7f35240536d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f354000b000 tx=0x7f3540006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:09.289 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.280+0000 7f3532ffd700 1 -- 172.21.15.117:0/2982232398 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3534042c50 con 0x7f3544141260 2020-11-20T13:56:09.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.480+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3544057050 con 0x7f3544141260 2020-11-20T13:56:09.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.481+0000 7f3532ffd700 1 -- 172.21.15.117:0/2982232398 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7f3534046270 con 0x7f3544141260 2020-11-20T13:56:09.483 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:09.483 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:55:51.676934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/1587825839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1587825839},{"type":"v1","addr":"172.21.15.117:6835","nonce":1587825839}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:09.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3524051210 msgr2=0x7f35240536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3524051210 0x7f35240536d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f354000b000 tx=0x7f3540006dc0).stop 2020-11-20T13:56:09.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3544141260 msgr2=0x7f354409ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:09.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3544141260 0x7f354409ec50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f3534002f30 tx=0x7f3534015000).stop 2020-11-20T13:56:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 shutdown_connections 2020-11-20T13:56:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3524051210 0x7f35240536d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f354414a830 0x7f354415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f354405cd80 0x7f3544146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.486 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 --2- 172.21.15.117:0/2982232398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3544141260 0x7f354409ec50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 >> 172.21.15.117:0/2982232398 conn(0x7f35440b7480 msgr2=0x7f3544060a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 shutdown_connections 2020-11-20T13:56:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:09.482+0000 7f354be62700 1 -- 172.21.15.117:0/2982232398 wait complete. 2020-11-20T13:56:09.487 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2020-11-20T13:56:09.695 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:55:50.675265+0000', 'modified': '2020-11-20T13:55:51.676934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/1587825839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1587825839}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1587825839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:09.696 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:09.696 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:09.821 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":4288,"want_state":"up:active","state":"up:active","rank_uptime":19.136472409,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":25.526401643} 2020-11-20T13:56:09.822 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 4288, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 19.136472409, "state": "up:active", "uptime": 25.526401643, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:09.822 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:09.823 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2020-11-20T13:56:09.823 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2020-11-20T13:56:09.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:09.824 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:09.842 INFO:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2020-11-20T13:56:09.937 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:09.938 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:09.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:09.969 INFO:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2020-11-20T13:56:10.024 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-20T13:56:10.024 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:10.066 INFO:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2020-11-20T13:56:10.094 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:10.095 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:10.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi085.stderr:2020-11-20T13:56:10.141+0000 7fdcecb5b1c0 -1 init, newargv = 0x55bf3a24b6f0 newargc=9 2020-11-20T13:56:10.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi085.stderr:ceph-fuse[23839]: starting ceph client 2020-11-20T13:56:10.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi085.stderr:ceph-fuse[23839]: starting fuse 2020-11-20T13:56:10.163 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:10.164 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:10.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:10.165 INFO:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2020-11-20T13:56:10.218 INFO:teuthology.orchestra.run.smithi085.stdout:47 2020-11-20T13:56:10.218 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-20T13:56:10.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:10.219 INFO:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2020-11-20T13:56:10.219 INFO:teuthology.orchestra.run.smithi085:> import glob 2020-11-20T13:56:10.219 INFO:teuthology.orchestra.run.smithi085:> import re 2020-11-20T13:56:10.220 INFO:teuthology.orchestra.run.smithi085:> import os 2020-11-20T13:56:10.220 INFO:teuthology.orchestra.run.smithi085:> import subprocess 2020-11-20T13:56:10.220 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:10.221 INFO:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2020-11-20T13:56:10.221 INFO:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2020-11-20T13:56:10.222 INFO:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2020-11-20T13:56:10.222 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:10.222 INFO:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2020-11-20T13:56:10.223 INFO:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2020-11-20T13:56:10.223 INFO:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2020-11-20T13:56:10.223 INFO:teuthology.orchestra.run.smithi085:> return files[0] 2020-11-20T13:56:10.224 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:10.224 INFO:teuthology.orchestra.run.smithi085:> for f in files: 2020-11-20T13:56:10.224 INFO:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:10.225 INFO:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:10.225 INFO:teuthology.orchestra.run.smithi085:> return f 2020-11-20T13:56:10.226 INFO:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:10.226 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:10.226 INFO:teuthology.orchestra.run.smithi085:> print(find_socket("client.2")) 2020-11-20T13:56:10.226 INFO:teuthology.orchestra.run.smithi085:> ' 2020-11-20T13:56:10.308 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.2.23839.asok 2020-11-20T13:56:10.312 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.23839.asok 2020-11-20T13:56:10.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:10.313 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.23839.asok status 2020-11-20T13:56:10.416 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2020-11-20T13:56:10.417 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2020-11-20T13:56:10.417 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:10.417 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:10.417 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "2", 2020-11-20T13:56:10.417 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2020-11-20T13:56:10.418 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2020-11-20T13:56:10.418 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "23839", 2020-11-20T13:56:10.418 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2020-11-20T13:56:10.418 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:10.419 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2020-11-20T13:56:10.419 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:10.419 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 4478, 2020-11-20T13:56:10.419 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2020-11-20T13:56:10.419 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2020-11-20T13:56:10.419 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2020-11-20T13:56:10.420 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 4478 2020-11-20T13:56:10.420 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:10.420 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2020-11-20T13:56:10.420 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2020-11-20T13:56:10.421 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2020-11-20T13:56:10.421 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3607336014 2020-11-20T13:56:10.421 INFO:teuthology.orchestra.run.smithi085.stdout: } 2020-11-20T13:56:10.422 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:10.422 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2020-11-20T13:56:10.422 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2020-11-20T13:56:10.422 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2020-11-20T13:56:10.424 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3607336014 2020-11-20T13:56:10.424 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:10.424 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.4478 172.21.15.85:0/3607336014", 2020-11-20T13:56:10.425 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3607336014", 2020-11-20T13:56:10.425 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2020-11-20T13:56:10.425 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 5, 2020-11-20T13:56:10.425 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 25, 2020-11-20T13:56:10.425 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:10.426 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false 2020-11-20T13:56:10.426 INFO:teuthology.orchestra.run.smithi085.stdout:} 2020-11-20T13:56:10.427 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:10.537 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.535+0000 7fec9653b700 1 Processor -- start 2020-11-20T13:56:10.537 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.535+0000 7fec9653b700 1 -- start start 2020-11-20T13:56:10.540 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec9653b700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fec90098e30 0x7fec9009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec9653b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec9009b700 0x7fec901438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec9653b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec90143dd0 0x7fec900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec9653b700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec900b5510 con 0x7fec90098e30 2020-11-20T13:56:10.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec9653b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec900b57b0 con 0x7fec90143dd0 2020-11-20T13:56:10.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec9653b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900b5a50 con 0x7fec9009b700 2020-11-20T13:56:10.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec94ad8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec90143dd0 0x7fec900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec94ad8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec90143dd0 0x7fec900b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40896/0 (socket says 172.21.15.117:40896) 2020-11-20T13:56:10.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec94ad8700 1 -- 172.21.15.117:0/3805410847 learned_addr learned my addr 172.21.15.117:0/3805410847 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:10.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec8ffff700 1 --2- 172.21.15.117:0/3805410847 >> v2:172.21.15.85:3300/0 conn(0x7fec90098e30 0x7fec9009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.536+0000 7fec8f7fe700 1 --2- 172.21.15.117:0/3805410847 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec9009b700 0x7fec901438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec94ad8700 1 -- 172.21.15.117:0/3805410847 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec9009b700 msgr2=0x7fec901438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:10.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec94ad8700 1 --2- 172.21.15.117:0/3805410847 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec9009b700 0x7fec901438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec94ad8700 1 -- 172.21.15.117:0/3805410847 >> v2:172.21.15.85:3300/0 conn(0x7fec90098e30 msgr2=0x7fec9009b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:10.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec94ad8700 1 --2- 172.21.15.117:0/3805410847 >> v2:172.21.15.85:3300/0 conn(0x7fec90098e30 0x7fec9009b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec94ad8700 1 -- 172.21.15.117:0/3805410847 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90099210 con 0x7fec90143dd0 2020-11-20T13:56:10.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec94ad8700 1 --2- 172.21.15.117:0/3805410847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90143dd0 0x7fec900b60e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fec880032b0 tx=0x7fec88009270).ready entity=mon.1 client_cookie=e327c62f03eedd5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec8effd700 1 -- 172.21.15.117:0/3805410847 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec8800bc00 con 0x7fec90143dd0 2020-11-20T13:56:10.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec8effd700 1 -- 172.21.15.117:0/3805410847 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8800bd60 con 0x7fec90143dd0 2020-11-20T13:56:10.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec8effd700 1 -- 172.21.15.117:0/3805410847 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec88011400 con 0x7fec90143dd0 2020-11-20T13:56:10.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 -- 172.21.15.117:0/3805410847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90143dd0 msgr2=0x7fec900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 --2- 172.21.15.117:0/3805410847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90143dd0 0x7fec900b60e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fec880032b0 tx=0x7fec88009270).stop 2020-11-20T13:56:10.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 -- 172.21.15.117:0/3805410847 shutdown_connections 2020-11-20T13:56:10.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 --2- 172.21.15.117:0/3805410847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90143dd0 0x7fec900b60e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 --2- 172.21.15.117:0/3805410847 >> v2:172.21.15.85:3300/0 conn(0x7fec90098e30 0x7fec9009b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 --2- 172.21.15.117:0/3805410847 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec9009b700 0x7fec901438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 -- 172.21.15.117:0/3805410847 >> 172.21.15.117:0/3805410847 conn(0x7fec9005b9f0 msgr2=0x7fec9005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:10.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 -- 172.21.15.117:0/3805410847 shutdown_connections 2020-11-20T13:56:10.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.537+0000 7fec9653b700 1 -- 172.21.15.117:0/3805410847 wait complete. 2020-11-20T13:56:10.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.538+0000 7fec9653b700 1 Processor -- start 2020-11-20T13:56:10.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.538+0000 7fec9653b700 1 -- start start 2020-11-20T13:56:10.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec9653b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90098e30 0x7fec901340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec9653b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fec9009b700 0x7fec901392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec9653b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec90143dd0 0x7fec900b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec9653b700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec900b2480 con 0x7fec9009b700 2020-11-20T13:56:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec9653b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec900b2720 con 0x7fec90098e30 2020-11-20T13:56:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec9653b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900b29c0 con 0x7fec90143dd0 2020-11-20T13:56:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec8ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90098e30 0x7fec901340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec8ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90098e30 0x7fec901340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40900/0 (socket says 172.21.15.117:40900) 2020-11-20T13:56:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec8ffff700 1 -- 172.21.15.117:0/2540439670 learned_addr learned my addr 172.21.15.117:0/2540439670 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:10.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec94ad8700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec90143dd0 0x7fec900b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.539+0000 7fec8f7fe700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fec9009b700 0x7fec901392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8ffff700 1 -- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec90143dd0 msgr2=0x7fec900b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8ffff700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec90143dd0 0x7fec900b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8ffff700 1 -- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fec9009b700 msgr2=0x7fec901392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8ffff700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fec9009b700 0x7fec901392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8ffff700 1 -- 172.21.15.117:0/2540439670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec900b2170 con 0x7fec90098e30 2020-11-20T13:56:10.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8ffff700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90098e30 0x7fec901340d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fec80003070 tx=0x7fec80008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8d7fa700 1 -- 172.21.15.117:0/2540439670 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec80009790 con 0x7fec90098e30 2020-11-20T13:56:10.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8d7fa700 1 -- 172.21.15.117:0/2540439670 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec800098f0 con 0x7fec90098e30 2020-11-20T13:56:10.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec900b3350 con 0x7fec90098e30 2020-11-20T13:56:10.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec8d7fa700 1 -- 172.21.15.117:0/2540439670 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec80009c50 con 0x7fec90098e30 2020-11-20T13:56:10.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.540+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec900b0b00 con 0x7fec90098e30 2020-11-20T13:56:10.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.541+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec90057050 con 0x7fec90098e30 2020-11-20T13:56:10.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.541+0000 7fec8d7fa700 1 -- 172.21.15.117:0/2540439670 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fec80013480 con 0x7fec90098e30 2020-11-20T13:56:10.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.541+0000 7fec8d7fa700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fec70051210 0x7fec700536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.541+0000 7fec8d7fa700 1 -- 172.21.15.117:0/2540439670 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fec800696e0 con 0x7fec90098e30 2020-11-20T13:56:10.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.541+0000 7fec8f7fe700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fec70051210 0x7fec700536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.541+0000 7fec8f7fe700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fec70051210 0x7fec700536d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fec8400c000 tx=0x7fec84008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.545+0000 7fec8d7fa700 1 -- 172.21.15.117:0/2540439670 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fec800418a0 con 0x7fec90098e30 2020-11-20T13:56:10.697 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.695+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec9009f7d0 con 0x7fec90098e30 2020-11-20T13:56:10.697 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.696+0000 7fec8d7fa700 1 -- 172.21.15.117:0/2540439670 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fec80009db0 con 0x7fec90098e30 2020-11-20T13:56:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:10.698 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:10.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fec70051210 msgr2=0x7fec700536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fec70051210 0x7fec700536d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fec8400c000 tx=0x7fec84008040).stop 2020-11-20T13:56:10.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90098e30 msgr2=0x7fec901340d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90098e30 0x7fec901340d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fec80003070 tx=0x7fec80008040).stop 2020-11-20T13:56:10.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 shutdown_connections 2020-11-20T13:56:10.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fec70051210 0x7fec700536d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fec90143dd0 0x7fec900b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec90098e30 0x7fec901340d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 --2- 172.21.15.117:0/2540439670 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fec9009b700 0x7fec901392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.697+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 >> 172.21.15.117:0/2540439670 conn(0x7fec9005b9f0 msgr2=0x7fec9009bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:10.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.698+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 shutdown_connections 2020-11-20T13:56:10.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.698+0000 7fec9653b700 1 -- 172.21.15.117:0/2540439670 wait complete. 2020-11-20T13:56:10.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:10.710 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:10.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.817+0000 7f66ddd4c700 1 Processor -- start 2020-11-20T13:56:10.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.817+0000 7f66ddd4c700 1 -- start start 2020-11-20T13:56:10.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66ddd4c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 0x7f66d81437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66ddd4c700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80999d0 con 0x7f66d809bd80 2020-11-20T13:56:10.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66ddd4c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d8099c70 con 0x7f66d8143c90 2020-11-20T13:56:10.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66ddd4c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80997f0 con 0x7f66d809cb40 2020-11-20T13:56:10.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d7fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f66d8143c90 0x7f66d80b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60072/0 (socket says 172.21.15.117:60072) 2020-11-20T13:56:10.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d7fff700 1 -- 172.21.15.117:0/2927230955 learned_addr learned my addr 172.21.15.117:0/2927230955 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:10.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d6ffd700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 0x7f66d81437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470875008 0 0) 0x7f66d8099c70 con 0x7f66d8143c90 2020-11-20T13:56:10.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b8003230 con 0x7f66d8143c90 2020-11-20T13:56:10.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 197354691 0 0) 0x7f66b8003230 con 0x7f66d8143c90 2020-11-20T13:56:10.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 msgr2=0x7f66d81437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:10.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 0x7f66d81437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 >> v1:172.21.15.85:6789/0 conn(0x7f66d809bd80 legacy=0x7f66d809a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d8144120 con 0x7f66d8143c90 2020-11-20T13:56:10.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f66c80016d0 con 0x7f66d8143c90 2020-11-20T13:56:10.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 >> v1:172.21.15.117:6789/0 conn(0x7f66d8143c90 legacy=0x7f66d80b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8003750 0x7f66b8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 0x7f66b80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66b80049c0 0x7f66b8004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80997f0 con 0x7f66b80049c0 2020-11-20T13:56:10.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80041f0 con 0x7f66b8003750 2020-11-20T13:56:10.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b800b720 con 0x7f66d809cb40 2020-11-20T13:56:10.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d77fe700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8003750 0x7f66b8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d6ffd700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 0x7f66b80044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.818+0000 7f66d7fff700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66b80049c0 0x7f66b8004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d77fe700 1 -- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 msgr2=0x7f66b80044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:10.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d77fe700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d809cb40 0x7f66b80044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d77fe700 1 -- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66b80049c0 msgr2=0x7f66b8004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:10.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d77fe700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66b80049c0 0x7f66b8004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d77fe700 1 -- 172.21.15.117:0/2927230955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8003230 con 0x7f66b8003750 2020-11-20T13:56:10.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d7fff700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66b80049c0 0x7f66b8004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:10.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d77fe700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8003750 0x7f66b8003b70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f66b80054a0 tx=0x7f66d0007e50).ready entity=mon.1 client_cookie=b8260436af2826fb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d000ca10 con 0x7f66b8003750 2020-11-20T13:56:10.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d000cb70 con 0x7f66b8003750 2020-11-20T13:56:10.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d67fc700 1 -- 172.21.15.117:0/2927230955 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66d0012400 con 0x7f66b8003750 2020-11-20T13:56:10.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66ddd4c700 1 -- 172.21.15.117:0/2927230955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8003750 msgr2=0x7f66b8003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66ddd4c700 1 --2- 172.21.15.117:0/2927230955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66b8003750 0x7f66b8003b70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f66b80054a0 tx=0x7f66d0007e50).stop 2020-11-20T13:56:10.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66d7fff700 1 -- 172.21.15.117:0/2927230955 reap_dead start 2020-11-20T13:56:10.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66ddd4c700 1 -- 172.21.15.117:0/2927230955 shutdown_connections 2020-11-20T13:56:10.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66ddd4c700 1 -- 172.21.15.117:0/2927230955 >> 172.21.15.117:0/2927230955 conn(0x7f66d80b9770 msgr2=0x7f66d80b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:10.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66ddd4c700 1 -- 172.21.15.117:0/2927230955 shutdown_connections 2020-11-20T13:56:10.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.819+0000 7f66ddd4c700 1 -- 172.21.15.117:0/2927230955 wait complete. 2020-11-20T13:56:10.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.820+0000 7f66ddd4c700 1 Processor -- start 2020-11-20T13:56:10.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66ddd4c700 1 -- start start 2020-11-20T13:56:10.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66ddd4c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66d809e390 0x7f66d8156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66ddd4c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d80671b0 0x7f66d8067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66ddd4c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d81408d0 0x7f66d8140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66ddd4c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8141240 con 0x7f66d809e390 2020-11-20T13:56:10.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66ddd4c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8156510 con 0x7f66d80671b0 2020-11-20T13:56:10.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66ddd4c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81567b0 con 0x7f66d81408d0 2020-11-20T13:56:10.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66d6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d80671b0 0x7f66d8067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66d6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d80671b0 0x7f66d8067630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40922/0 (socket says 172.21.15.117:40922) 2020-11-20T13:56:10.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66d6ffd700 1 -- 172.21.15.117:0/1723064981 learned_addr learned my addr 172.21.15.117:0/1723064981 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:10.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66d77fe700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66d809e390 0x7f66d8156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.821+0000 7f66d7fff700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d81408d0 0x7f66d8140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d6ffd700 1 -- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d81408d0 msgr2=0x7f66d8140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d6ffd700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d81408d0 0x7f66d8140d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d6ffd700 1 -- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66d809e390 msgr2=0x7f66d8156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d6ffd700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66d809e390 0x7f66d8156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d6ffd700 1 -- 172.21.15.117:0/1723064981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d8156a50 con 0x7f66d80671b0 2020-11-20T13:56:10.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d6ffd700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d80671b0 0x7f66d8067630 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f66c0006750 tx=0x7f66c0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d4ff9700 1 -- 172.21.15.117:0/1723064981 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c000fa90 con 0x7f66d80671b0 2020-11-20T13:56:10.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d4ff9700 1 -- 172.21.15.117:0/1723064981 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c000fbf0 con 0x7f66d80671b0 2020-11-20T13:56:10.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66d4ff9700 1 -- 172.21.15.117:0/1723064981 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66c000fd60 con 0x7f66d80671b0 2020-11-20T13:56:10.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d809e770 con 0x7f66d80671b0 2020-11-20T13:56:10.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.822+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d8152fc0 con 0x7f66d80671b0 2020-11-20T13:56:10.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.823+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d8156be0 con 0x7f66d80671b0 2020-11-20T13:56:10.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.823+0000 7f66d4ff9700 1 -- 172.21.15.117:0/1723064981 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f66c0024070 con 0x7f66d80671b0 2020-11-20T13:56:10.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.823+0000 7f66d4ff9700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66bc051320 0x7f66bc0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:10.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.823+0000 7f66d4ff9700 1 -- 172.21.15.117:0/1723064981 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f66c0069a50 con 0x7f66d80671b0 2020-11-20T13:56:10.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.823+0000 7f66d77fe700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66bc051320 0x7f66bc0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:10.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.823+0000 7f66d77fe700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66bc051320 0x7f66bc0537e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f66d001c000 tx=0x7f66d0015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:10.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.827+0000 7f66d4ff9700 1 -- 172.21.15.117:0/1723064981 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f66c006be60 con 0x7f66d80671b0 2020-11-20T13:56:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.975+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f66d8057050 con 0x7f66d80671b0 2020-11-20T13:56:10.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.975+0000 7f66d4ff9700 1 -- 172.21.15.117:0/1723064981 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f66c0041c70 con 0x7f66d80671b0 2020-11-20T13:56:10.977 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:10.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66bc051320 msgr2=0x7f66bc0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66bc051320 0x7f66bc0537e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f66d001c000 tx=0x7f66d0015000).stop 2020-11-20T13:56:10.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d80671b0 msgr2=0x7f66d8067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:10.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d80671b0 0x7f66d8067630 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f66c0006750 tx=0x7f66c0008040).stop 2020-11-20T13:56:10.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 shutdown_connections 2020-11-20T13:56:10.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f66bc051320 0x7f66bc0537e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f66d81408d0 0x7f66d8140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f66d809e390 0x7f66d8156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 --2- 172.21.15.117:0/1723064981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66d80671b0 0x7f66d8067630 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:10.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 >> 172.21.15.117:0/1723064981 conn(0x7f66d80b9770 msgr2=0x7f66d8142a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:10.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 shutdown_connections 2020-11-20T13:56:10.981 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:10.977+0000 7f66ddd4c700 1 -- 172.21.15.117:0/1723064981 wait complete. 2020-11-20T13:56:10.988 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:11.098 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.096+0000 7fb21e743700 1 Processor -- start 2020-11-20T13:56:11.098 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.096+0000 7fb21e743700 1 -- start start 2020-11-20T13:56:11.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb21e743700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb218062910 0x7fb21809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb21e743700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb218062cf0 0x7fb21805f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb21e743700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21809b3c0 con 0x7fb21809af30 2020-11-20T13:56:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb21e743700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21809b660 con 0x7fb218062cf0 2020-11-20T13:56:11.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb21e743700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21809b900 con 0x7fb218062910 2020-11-20T13:56:11.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb217fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb218062910 0x7fb21809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb21cce0700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb21809af30 0x7fb218097f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60090/0 (socket says 172.21.15.117:60090) 2020-11-20T13:56:11.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.097+0000 7fb21cce0700 1 -- 172.21.15.117:0/218910953 learned_addr learned my addr 172.21.15.117:0/218910953 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:11.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb20ffff700 1 --2- 172.21.15.117:0/218910953 >> v2:172.21.15.85:3300/0 conn(0x7fb218062cf0 0x7fb21805f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb217fff700 1 -- 172.21.15.117:0/218910953 >> v2:172.21.15.85:3300/0 conn(0x7fb218062cf0 msgr2=0x7fb21805f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb217fff700 1 --2- 172.21.15.117:0/218910953 >> v2:172.21.15.85:3300/0 conn(0x7fb218062cf0 0x7fb21805f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb217fff700 1 -- 172.21.15.117:0/218910953 >> v1:172.21.15.117:6789/0 conn(0x7fb21809af30 legacy=0x7fb218097f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb217fff700 1 -- 172.21.15.117:0/218910953 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180b4310 con 0x7fb218062910 2020-11-20T13:56:11.103 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb2177fe700 1 -- 172.21.15.117:0/218910953 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322333299 0 0) 0x7fb21809b3c0 con 0x7fb21809af30 2020-11-20T13:56:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb20ffff700 1 --2- 172.21.15.117:0/218910953 >> v2:172.21.15.85:3300/0 conn(0x7fb218062cf0 0x7fb21805f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb217fff700 1 --2- 172.21.15.117:0/218910953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062910 0x7fb21809aa40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb2180ba330 tx=0x7fb208006dc0).ready entity=mon.1 client_cookie=383e517d25b783c5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.104 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb2177fe700 1 -- 172.21.15.117:0/218910953 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2080097b0 con 0x7fb218062910 2020-11-20T13:56:11.105 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb2177fe700 1 -- 172.21.15.117:0/218910953 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb208009910 con 0x7fb218062910 2020-11-20T13:56:11.105 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb2177fe700 1 -- 172.21.15.117:0/218910953 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb208009c70 con 0x7fb218062910 2020-11-20T13:56:11.105 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb21e743700 1 -- 172.21.15.117:0/218910953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062910 msgr2=0x7fb21809aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.106 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb21e743700 1 --2- 172.21.15.117:0/218910953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062910 0x7fb21809aa40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb2180ba330 tx=0x7fb208006dc0).stop 2020-11-20T13:56:11.106 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb21e743700 1 -- 172.21.15.117:0/218910953 shutdown_connections 2020-11-20T13:56:11.106 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb21e743700 1 --2- 172.21.15.117:0/218910953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062910 0x7fb21809aa40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.106 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb21e743700 1 --2- 172.21.15.117:0/218910953 >> v2:172.21.15.85:3300/0 conn(0x7fb218062cf0 0x7fb21805f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.098+0000 7fb21e743700 1 -- 172.21.15.117:0/218910953 >> 172.21.15.117:0/218910953 conn(0x7fb2180b9770 msgr2=0x7fb2180b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.099+0000 7fb21e743700 1 -- 172.21.15.117:0/218910953 shutdown_connections 2020-11-20T13:56:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.099+0000 7fb21e743700 1 -- 172.21.15.117:0/218910953 wait complete. 2020-11-20T13:56:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.099+0000 7fb21e743700 1 Processor -- start 2020-11-20T13:56:11.107 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.099+0000 7fb21e743700 1 -- start start 2020-11-20T13:56:11.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb21e743700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb218062910 0x7fb218067230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb21e743700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062cf0 0x7fb218067720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb21e743700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2180aebf0 0x7fb218148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb21e743700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b10d0 con 0x7fb2180aebf0 2020-11-20T13:56:11.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb21e743700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b12c0 con 0x7fb218062cf0 2020-11-20T13:56:11.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb21e743700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180b1560 con 0x7fb218062910 2020-11-20T13:56:11.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062cf0 0x7fb218067720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062cf0 0x7fb218067720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40936/0 (socket says 172.21.15.117:40936) 2020-11-20T13:56:11.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 -- 172.21.15.117:0/3045091994 learned_addr learned my addr 172.21.15.117:0/3045091994 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:11.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb217fff700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb218062910 0x7fb218067230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb21cce0700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2180aebf0 0x7fb218148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 -- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb218062910 msgr2=0x7fb218067230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb218062910 0x7fb218067230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 -- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2180aebf0 msgr2=0x7fb218148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.113 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2180aebf0 0x7fb218148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.113 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.100+0000 7fb20ffff700 1 -- 172.21.15.117:0/3045091994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180b1800 con 0x7fb218062cf0 2020-11-20T13:56:11.113 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.101+0000 7fb20ffff700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062cf0 0x7fb218067720 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fb2180af610 tx=0x7fb200008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.113 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.101+0000 7fb215ffb700 1 -- 172.21.15.117:0/3045091994 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb200009920 con 0x7fb218062cf0 2020-11-20T13:56:11.113 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.101+0000 7fb215ffb700 1 -- 172.21.15.117:0/3045091994 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb200009a80 con 0x7fb218062cf0 2020-11-20T13:56:11.114 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.101+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb218148aa0 con 0x7fb218062cf0 2020-11-20T13:56:11.114 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.101+0000 7fb215ffb700 1 -- 172.21.15.117:0/3045091994 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb200009de0 con 0x7fb218062cf0 2020-11-20T13:56:11.114 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.101+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218148fc0 con 0x7fb218062cf0 2020-11-20T13:56:11.114 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.102+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb218057050 con 0x7fb218062cf0 2020-11-20T13:56:11.114 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.102+0000 7fb215ffb700 1 -- 172.21.15.117:0/3045091994 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb200013480 con 0x7fb218062cf0 2020-11-20T13:56:11.115 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.102+0000 7fb215ffb700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb1fc051320 0x7fb1fc0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.115 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.102+0000 7fb215ffb700 1 -- 172.21.15.117:0/3045091994 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb200069800 con 0x7fb218062cf0 2020-11-20T13:56:11.115 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.102+0000 7fb217fff700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb1fc051320 0x7fb1fc0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.115 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.103+0000 7fb217fff700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb1fc051320 0x7fb1fc0537e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb208019000 tx=0x7fb208015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.115 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.106+0000 7fb215ffb700 1 -- 172.21.15.117:0/3045091994 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb2000419c0 con 0x7fb218062cf0 2020-11-20T13:56:11.258 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.257+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb21809a920 con 0x7fb218062cf0 2020-11-20T13:56:11.258 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.257+0000 7fb215ffb700 1 -- 172.21.15.117:0/3045091994 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb200044fe0 con 0x7fb218062cf0 2020-11-20T13:56:11.259 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:11.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb1fc051320 msgr2=0x7fb1fc0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb1fc051320 0x7fb1fc0537e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb208019000 tx=0x7fb208015000).stop 2020-11-20T13:56:11.261 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062cf0 msgr2=0x7fb218067720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062cf0 0x7fb218067720 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fb2180af610 tx=0x7fb200008040).stop 2020-11-20T13:56:11.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 shutdown_connections 2020-11-20T13:56:11.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb1fc051320 0x7fb1fc0537e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb218062910 0x7fb218067230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2180aebf0 0x7fb218148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 --2- 172.21.15.117:0/3045091994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb218062cf0 0x7fb218067720 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 >> 172.21.15.117:0/3045091994 conn(0x7fb2180b9770 msgr2=0x7fb2180b76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:11.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 shutdown_connections 2020-11-20T13:56:11.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.258+0000 7fb21e743700 1 -- 172.21.15.117:0/3045091994 wait complete. 2020-11-20T13:56:11.271 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:11.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.378+0000 7f36ab4b6700 1 Processor -- start 2020-11-20T13:56:11.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.378+0000 7f36ab4b6700 1 -- start start 2020-11-20T13:56:11.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36ab4b6700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f36a405f300 0x7f36a4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36ab4b6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a4147030 0x7f36a409c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36ab4b6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a409f100 con 0x7f36a409bf70 2020-11-20T13:56:11.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36ab4b6700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a409eef0 con 0x7f36a405f300 2020-11-20T13:56:11.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36ab4b6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a409f3a0 con 0x7f36a4147030 2020-11-20T13:56:11.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36a9cb3700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f36a409bf70 0x7f36a4146920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60102/0 (socket says 172.21.15.117:60102) 2020-11-20T13:56:11.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36a9cb3700 1 -- 172.21.15.117:0/1103183758 learned_addr learned my addr 172.21.15.117:0/1103183758 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:11.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36aacb5700 1 --2- 172.21.15.117:0/1103183758 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a4147030 0x7f36a409c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.379+0000 7f36aa4b4700 1 --2- 172.21.15.117:0/1103183758 >> v2:172.21.15.85:3300/0 conn(0x7f36a405f300 0x7f36a4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.380+0000 7f36aacb5700 1 -- 172.21.15.117:0/1103183758 >> v2:172.21.15.85:3300/0 conn(0x7f36a405f300 msgr2=0x7f36a4062b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:11.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.380+0000 7f36aacb5700 1 --2- 172.21.15.117:0/1103183758 >> v2:172.21.15.85:3300/0 conn(0x7f36a405f300 0x7f36a4062b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.380+0000 7f36aacb5700 1 -- 172.21.15.117:0/1103183758 >> v1:172.21.15.117:6789/0 conn(0x7f36a409bf70 legacy=0x7f36a4146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.380+0000 7f36aacb5700 1 -- 172.21.15.117:0/1103183758 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a405f6e0 con 0x7f36a4147030 2020-11-20T13:56:11.384 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.380+0000 7f36aacb5700 1 --2- 172.21.15.117:0/1103183758 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a4147030 0x7f36a409c660 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f369c0032b0 tx=0x7f369c0091a0).ready entity=mon.2 client_cookie=535541a4373d8321 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.389 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.381+0000 7f36a94b2700 1 -- 172.21.15.117:0/1103183758 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f369c00b9a0 con 0x7f36a4147030 2020-11-20T13:56:11.389 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.381+0000 7f36a94b2700 1 -- 172.21.15.117:0/1103183758 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c00bb00 con 0x7f36a4147030 2020-11-20T13:56:11.389 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.381+0000 7f36a94b2700 1 -- 172.21.15.117:0/1103183758 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f369c00bc70 con 0x7f36a4147030 2020-11-20T13:56:11.389 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.381+0000 7f36ab4b6700 1 -- 172.21.15.117:0/1103183758 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a4147030 msgr2=0x7f36a409c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.381+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/1103183758 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a4147030 0x7f36a409c660 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f369c0032b0 tx=0x7f369c0091a0).stop 2020-11-20T13:56:11.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.382+0000 7f36ab4b6700 1 -- 172.21.15.117:0/1103183758 shutdown_connections 2020-11-20T13:56:11.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.382+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/1103183758 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a4147030 0x7f36a409c660 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.382+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/1103183758 >> v2:172.21.15.85:3300/0 conn(0x7f36a405f300 0x7f36a4062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.382+0000 7f36ab4b6700 1 -- 172.21.15.117:0/1103183758 >> 172.21.15.117:0/1103183758 conn(0x7f36a40b9490 msgr2=0x7f36a40b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:11.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.382+0000 7f36ab4b6700 1 -- 172.21.15.117:0/1103183758 shutdown_connections 2020-11-20T13:56:11.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.382+0000 7f36ab4b6700 1 -- 172.21.15.117:0/1103183758 wait complete. 2020-11-20T13:56:11.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 Processor -- start 2020-11-20T13:56:11.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 -- start start 2020-11-20T13:56:11.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a405f300 0x7f36a40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.392 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a409bf70 0x7f36a41577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.392 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36a4147030 0x7f36a4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.392 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40922b0 con 0x7f36a4147030 2020-11-20T13:56:11.392 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4092510 con 0x7f36a405f300 2020-11-20T13:56:11.392 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.383+0000 7f36ab4b6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40927b0 con 0x7f36a409bf70 2020-11-20T13:56:11.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a405f300 0x7f36a40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a405f300 0x7f36a40b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40948/0 (socket says 172.21.15.117:40948) 2020-11-20T13:56:11.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 -- 172.21.15.117:0/698883649 learned_addr learned my addr 172.21.15.117:0/698883649 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:11.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36a9cb3700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a409bf70 0x7f36a41577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aacb5700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36a4147030 0x7f36a4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 -- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a409bf70 msgr2=0x7f36a41577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a409bf70 0x7f36a41577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 -- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36a4147030 msgr2=0x7f36a4157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36a4147030 0x7f36a4157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 -- 172.21.15.117:0/698883649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a408feb0 con 0x7f36a405f300 2020-11-20T13:56:11.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36aa4b4700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a405f300 0x7f36a40b2ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f36a000c000 tx=0x7f36a0006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36937fe700 1 -- 172.21.15.117:0/698883649 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a000ca10 con 0x7f36a405f300 2020-11-20T13:56:11.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36937fe700 1 -- 172.21.15.117:0/698883649 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a000cb70 con 0x7f36a405f300 2020-11-20T13:56:11.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a4090160 con 0x7f36a405f300 2020-11-20T13:56:11.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36937fe700 1 -- 172.21.15.117:0/698883649 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36a00123f0 con 0x7f36a405f300 2020-11-20T13:56:11.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.384+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a41583d0 con 0x7f36a405f300 2020-11-20T13:56:11.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.385+0000 7f36937fe700 1 -- 172.21.15.117:0/698883649 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f36a0012550 con 0x7f36a405f300 2020-11-20T13:56:11.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.385+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a4057050 con 0x7f36a405f300 2020-11-20T13:56:11.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.385+0000 7f36937fe700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3684051290 0x7f3684053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.386+0000 7f36937fe700 1 -- 172.21.15.117:0/698883649 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f36a00693e0 con 0x7f36a405f300 2020-11-20T13:56:11.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.386+0000 7f36a9cb3700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3684051290 0x7f3684053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.386+0000 7f36a9cb3700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3684051290 0x7f3684053750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f369400d000 tx=0x7f3694009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.389+0000 7f36937fe700 1 -- 172.21.15.117:0/698883649 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f36a00415a0 con 0x7f36a405f300 2020-11-20T13:56:11.540 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.539+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36a4092970 con 0x7f36a405f300 2020-11-20T13:56:11.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.539+0000 7f36937fe700 1 -- 172.21.15.117:0/698883649 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f36a0044bc0 con 0x7f36a405f300 2020-11-20T13:56:11.541 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:11.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.540+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3684051290 msgr2=0x7f3684053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3684051290 0x7f3684053750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f369400d000 tx=0x7f3694009040).stop 2020-11-20T13:56:11.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a405f300 msgr2=0x7f36a40b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a405f300 0x7f36a40b2ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f36a000c000 tx=0x7f36a0006dc0).stop 2020-11-20T13:56:11.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 shutdown_connections 2020-11-20T13:56:11.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3684051290 0x7f3684053750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36a4147030 0x7f36a4157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a405f300 0x7f36a40b2ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 --2- 172.21.15.117:0/698883649 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36a409bf70 0x7f36a41577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 >> 172.21.15.117:0/698883649 conn(0x7f36a40b9490 msgr2=0x7f36a4066970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:11.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 shutdown_connections 2020-11-20T13:56:11.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.541+0000 7f36ab4b6700 1 -- 172.21.15.117:0/698883649 wait complete. 2020-11-20T13:56:11.552 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:11.661 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.660+0000 7f39e121e700 1 Processor -- start 2020-11-20T13:56:11.661 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.660+0000 7f39e121e700 1 -- start start 2020-11-20T13:56:11.663 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.660+0000 7f39e121e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09bd80 0x7f39dc09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.660+0000 7f39e121e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f39dc09a110 0x7f39dc1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.660+0000 7f39e121e700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39dc0999d0 con 0x7f39dc143c90 2020-11-20T13:56:11.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39e121e700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39dc099c70 con 0x7f39dc09a110 2020-11-20T13:56:11.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39e121e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc144120 con 0x7f39dc09bd80 2020-11-20T13:56:11.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f39dc09a110 0x7f39dc1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39dad9d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09bd80 0x7f39dc09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f39dc09a110 0x7f39dc1437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:52582/0 (socket says 172.21.15.117:52582) 2020-11-20T13:56:11.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 -- 172.21.15.117:0/1616782495 learned_addr learned my addr 172.21.15.117:0/1616782495 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:11.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 -- 172.21.15.117:0/1616782495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09bd80 msgr2=0x7f39dc09cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:11.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 --2- 172.21.15.117:0/1616782495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09bd80 0x7f39dc09cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 -- 172.21.15.117:0/1616782495 >> v1:172.21.15.85:6789/0 conn(0x7f39dc143c90 legacy=0x7f39dc0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 -- 172.21.15.117:0/1616782495 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc09c160 con 0x7f39dc09a110 2020-11-20T13:56:11.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39da59c700 1 --2- 172.21.15.117:0/1616782495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09a110 0x7f39dc1437a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f39dc09b9c0 tx=0x7f39c4006e80).ready entity=mon.0 client_cookie=aa6b5617301763a5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.661+0000 7f39d9d9b700 1 -- 172.21.15.117:0/1616782495 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c40097b0 con 0x7f39dc09a110 2020-11-20T13:56:11.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39d9d9b700 1 -- 172.21.15.117:0/1616782495 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4009910 con 0x7f39dc09a110 2020-11-20T13:56:11.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39d9d9b700 1 -- 172.21.15.117:0/1616782495 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c4009c70 con 0x7f39dc09a110 2020-11-20T13:56:11.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 -- 172.21.15.117:0/1616782495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09a110 msgr2=0x7f39dc1437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 --2- 172.21.15.117:0/1616782495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09a110 0x7f39dc1437a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f39dc09b9c0 tx=0x7f39c4006e80).stop 2020-11-20T13:56:11.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 -- 172.21.15.117:0/1616782495 shutdown_connections 2020-11-20T13:56:11.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 --2- 172.21.15.117:0/1616782495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09bd80 0x7f39dc09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 --2- 172.21.15.117:0/1616782495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09a110 0x7f39dc1437a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 -- 172.21.15.117:0/1616782495 >> 172.21.15.117:0/1616782495 conn(0x7f39dc0b9770 msgr2=0x7f39dc0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:11.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 -- 172.21.15.117:0/1616782495 shutdown_connections 2020-11-20T13:56:11.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.662+0000 7f39e121e700 1 -- 172.21.15.117:0/1616782495 wait complete. 2020-11-20T13:56:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.663+0000 7f39e121e700 1 Processor -- start 2020-11-20T13:56:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.663+0000 7f39e121e700 1 -- start start 2020-11-20T13:56:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39e121e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09a110 0x7f39dc0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39e121e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09bd80 0x7f39dc153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39e121e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39dc143c90 0x7f39dc0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39e121e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc153fb0 con 0x7f39dc09bd80 2020-11-20T13:56:11.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39e121e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc0b3250 con 0x7f39dc143c90 2020-11-20T13:56:11.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39e121e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc0b34f0 con 0x7f39dc09a110 2020-11-20T13:56:11.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39dc143c90 0x7f39dc0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39dc143c90 0x7f39dc0aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40966/0 (socket says 172.21.15.117:40966) 2020-11-20T13:56:11.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 -- 172.21.15.117:0/1689959545 learned_addr learned my addr 172.21.15.117:0/1689959545 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:11.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39dad9d700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09a110 0x7f39dc0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39da59c700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09bd80 0x7f39dc153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 -- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09a110 msgr2=0x7f39dc0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09a110 0x7f39dc0673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 -- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09bd80 msgr2=0x7f39dc153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09bd80 0x7f39dc153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 -- 172.21.15.117:0/1689959545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc0ae770 con 0x7f39dc143c90 2020-11-20T13:56:11.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39db59e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39dc143c90 0x7f39dc0aec10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f39dc09ad40 tx=0x7f39d4007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39d3fff700 1 -- 172.21.15.117:0/1689959545 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d4006860 con 0x7f39dc143c90 2020-11-20T13:56:11.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39d3fff700 1 -- 172.21.15.117:0/1689959545 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d40069c0 con 0x7f39dc143c90 2020-11-20T13:56:11.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc0b0150 con 0x7f39dc143c90 2020-11-20T13:56:11.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.664+0000 7f39d3fff700 1 -- 172.21.15.117:0/1689959545 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39d4006d20 con 0x7f39dc143c90 2020-11-20T13:56:11.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.665+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc0b2fe0 con 0x7f39dc143c90 2020-11-20T13:56:11.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.666+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc0ae900 con 0x7f39dc143c90 2020-11-20T13:56:11.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.666+0000 7f39d3fff700 1 -- 172.21.15.117:0/1689959545 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f39d4010070 con 0x7f39dc143c90 2020-11-20T13:56:11.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.666+0000 7f39d3fff700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39c00513c0 0x7f39c0053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.666+0000 7f39d3fff700 1 -- 172.21.15.117:0/1689959545 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f39d40688f0 con 0x7f39dc143c90 2020-11-20T13:56:11.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.666+0000 7f39dad9d700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39c00513c0 0x7f39c0053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.666+0000 7f39dad9d700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39c00513c0 0x7f39c0053880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f39cc008000 tx=0x7f39cc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.669+0000 7f39d3fff700 1 -- 172.21.15.117:0/1689959545 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f39d4040a10 con 0x7f39dc143c90 2020-11-20T13:56:11.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.820+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39dc057050 con 0x7f39dc143c90 2020-11-20T13:56:11.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.820+0000 7f39d3fff700 1 -- 172.21.15.117:0/1689959545 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f39d4044030 con 0x7f39dc143c90 2020-11-20T13:56:11.822 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:11.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39c00513c0 msgr2=0x7f39c0053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39c00513c0 0x7f39c0053880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f39cc008000 tx=0x7f39cc008040).stop 2020-11-20T13:56:11.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39dc143c90 msgr2=0x7f39dc0aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39dc143c90 0x7f39dc0aec10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f39dc09ad40 tx=0x7f39d4007eb0).stop 2020-11-20T13:56:11.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 shutdown_connections 2020-11-20T13:56:11.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39c00513c0 0x7f39c0053880 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f39dc09a110 0x7f39dc0673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39dc143c90 0x7f39dc0aec10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 --2- 172.21.15.117:0/1689959545 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f39dc09bd80 0x7f39dc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 >> 172.21.15.117:0/1689959545 conn(0x7f39dc0b9770 msgr2=0x7f39dc0b9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:11.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 shutdown_connections 2020-11-20T13:56:11.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.822+0000 7f39e121e700 1 -- 172.21.15.117:0/1689959545 wait complete. 2020-11-20T13:56:11.833 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:11.833 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:11.941 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.937+0000 7fc01912c700 1 Processor -- start 2020-11-20T13:56:11.941 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.937+0000 7fc01912c700 1 -- start start 2020-11-20T13:56:11.942 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.938+0000 7fc01912c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc014098e30 0x7fc01409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.942 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.938+0000 7fc01912c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fc01409b700 0x7fc0141438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.942 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.938+0000 7fc01912c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0140b5510 con 0x7fc014143dd0 2020-11-20T13:56:11.942 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.938+0000 7fc01912c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0140b57b0 con 0x7fc01409b700 2020-11-20T13:56:11.943 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.938+0000 7fc01912c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140b5a50 con 0x7fc014098e30 2020-11-20T13:56:11.943 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.938+0000 7fc01359e700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc014143dd0 0x7fc0140b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60130/0 (socket says 172.21.15.117:60130) 2020-11-20T13:56:11.943 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.938+0000 7fc01359e700 1 -- 172.21.15.117:0/188027514 learned_addr learned my addr 172.21.15.117:0/188027514 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:11.943 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc012d9d700 1 --2- 172.21.15.117:0/188027514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc014098e30 0x7fc01409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.944 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc01259c700 1 --2- 172.21.15.117:0/188027514 >> v2:172.21.15.85:3300/0 conn(0x7fc01409b700 0x7fc0141438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.944 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc011d9b700 1 -- 172.21.15.117:0/188027514 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228524069 0 0) 0x7fc0140b5510 con 0x7fc014143dd0 2020-11-20T13:56:11.944 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc011d9b700 1 -- 172.21.15.117:0/188027514 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff8003230 con 0x7fc014143dd0 2020-11-20T13:56:11.944 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc011d9b700 1 -- 172.21.15.117:0/188027514 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 723890149 0 0) 0x7fbff8003230 con 0x7fc014143dd0 2020-11-20T13:56:11.945 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc012d9d700 1 -- 172.21.15.117:0/188027514 >> v2:172.21.15.85:3300/0 conn(0x7fc01409b700 msgr2=0x7fc0141438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:11.945 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc012d9d700 1 --2- 172.21.15.117:0/188027514 >> v2:172.21.15.85:3300/0 conn(0x7fc01409b700 0x7fc0141438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.945 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc012d9d700 1 -- 172.21.15.117:0/188027514 >> v1:172.21.15.117:6789/0 conn(0x7fc014143dd0 legacy=0x7fc0140b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.945 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc012d9d700 1 -- 172.21.15.117:0/188027514 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014099210 con 0x7fc014098e30 2020-11-20T13:56:11.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc01259c700 1 --2- 172.21.15.117:0/188027514 >> v2:172.21.15.85:3300/0 conn(0x7fc01409b700 0x7fc0141438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:11.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc012d9d700 1 --2- 172.21.15.117:0/188027514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc014098e30 0x7fc01409b210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc00c001bf0 tx=0x7fc008006dc0).ready entity=mon.2 client_cookie=408bc60daec17135 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc011d9b700 1 -- 172.21.15.117:0/188027514 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc00c001a50 con 0x7fc014098e30 2020-11-20T13:56:11.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc011d9b700 1 -- 172.21.15.117:0/188027514 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00800d570 con 0x7fc014098e30 2020-11-20T13:56:11.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc011d9b700 1 -- 172.21.15.117:0/188027514 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc00800d6e0 con 0x7fc014098e30 2020-11-20T13:56:11.947 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc01912c700 1 -- 172.21.15.117:0/188027514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc014098e30 msgr2=0x7fc01409b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.939+0000 7fc01912c700 1 --2- 172.21.15.117:0/188027514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc014098e30 0x7fc01409b210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc00c001bf0 tx=0x7fc008006dc0).stop 2020-11-20T13:56:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.940+0000 7fc01912c700 1 -- 172.21.15.117:0/188027514 shutdown_connections 2020-11-20T13:56:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.940+0000 7fc01912c700 1 --2- 172.21.15.117:0/188027514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc014098e30 0x7fc01409b210 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.940+0000 7fc01912c700 1 --2- 172.21.15.117:0/188027514 >> v2:172.21.15.85:3300/0 conn(0x7fc01409b700 0x7fc0141438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.949 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.940+0000 7fc01912c700 1 -- 172.21.15.117:0/188027514 >> 172.21.15.117:0/188027514 conn(0x7fc01405b9f0 msgr2=0x7fc01405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:11.950 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.940+0000 7fc01912c700 1 -- 172.21.15.117:0/188027514 shutdown_connections 2020-11-20T13:56:11.950 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.940+0000 7fc01912c700 1 -- 172.21.15.117:0/188027514 wait complete. 2020-11-20T13:56:11.953 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 Processor -- start 2020-11-20T13:56:11.953 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 -- start start 2020-11-20T13:56:11.953 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc014098e30 0x7fc01409c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.953 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc01409b700 0x7fc0141459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.953 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc014143dd0 0x7fc014066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.954 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01409f1d0 con 0x7fc014143dd0 2020-11-20T13:56:11.954 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01409f470 con 0x7fc014098e30 2020-11-20T13:56:11.954 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.941+0000 7fc01912c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01409f710 con 0x7fc01409b700 2020-11-20T13:56:11.954 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc014098e30 0x7fc01409c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc014098e30 0x7fc01409c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40976/0 (socket says 172.21.15.117:40976) 2020-11-20T13:56:11.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 -- 172.21.15.117:0/1115563018 learned_addr learned my addr 172.21.15.117:0/1115563018 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:11.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc01259c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc01409b700 0x7fc0141459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc01359e700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc014143dd0 0x7fc014066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 -- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc01409b700 msgr2=0x7fc0141459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.956 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc01409b700 0x7fc0141459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.957 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 -- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc014143dd0 msgr2=0x7fc014066f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:11.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc014143dd0 0x7fc014066f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:11.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 -- 172.21.15.117:0/1115563018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0140b0af0 con 0x7fc014098e30 2020-11-20T13:56:11.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc012d9d700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc014098e30 0x7fc01409c7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fc00c001bf0 tx=0x7fc008015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc003fff700 1 -- 172.21.15.117:0/1115563018 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc008011890 con 0x7fc014098e30 2020-11-20T13:56:11.958 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc003fff700 1 -- 172.21.15.117:0/1115563018 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0080119f0 con 0x7fc014098e30 2020-11-20T13:56:11.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0140b3470 con 0x7fc014098e30 2020-11-20T13:56:11.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc003fff700 1 -- 172.21.15.117:0/1115563018 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc008011d50 con 0x7fc014098e30 2020-11-20T13:56:11.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.942+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01413eab0 con 0x7fc014098e30 2020-11-20T13:56:11.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.943+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01409f880 con 0x7fc014098e30 2020-11-20T13:56:11.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.943+0000 7fc003fff700 1 -- 172.21.15.117:0/1115563018 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc008024030 con 0x7fc014098e30 2020-11-20T13:56:11.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.944+0000 7fc003fff700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fbff0051210 0x7fbff00536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:11.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.944+0000 7fc003fff700 1 -- 172.21.15.117:0/1115563018 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc008069d80 con 0x7fc014098e30 2020-11-20T13:56:11.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.944+0000 7fc01259c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fbff0051210 0x7fbff00536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:11.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.944+0000 7fc01259c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fbff0051210 0x7fbff00536d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc004008000 tx=0x7fc004008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:11.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:11.947+0000 7fc003fff700 1 -- 172.21.15.117:0/1115563018 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc01409f880 con 0x7fc014098e30 2020-11-20T13:56:12.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.145+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc01409f880 con 0x7fc014098e30 2020-11-20T13:56:12.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.146+0000 7fc003fff700 1 -- 172.21.15.117:0/1115563018 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7fc01409f880 con 0x7fc014098e30 2020-11-20T13:56:12.147 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:12.148 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:55:51.676934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/1587825839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1587825839},{"type":"v1","addr":"172.21.15.117:6835","nonce":1587825839}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:12.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fbff0051210 msgr2=0x7fbff00536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:12.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fbff0051210 0x7fbff00536d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc004008000 tx=0x7fc004008040).stop 2020-11-20T13:56:12.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc014098e30 msgr2=0x7fc01409c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:12.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc014098e30 0x7fc01409c7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fc00c001bf0 tx=0x7fc008015000).stop 2020-11-20T13:56:12.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 shutdown_connections 2020-11-20T13:56:12.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fbff0051210 0x7fbff00536d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:12.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc014143dd0 0x7fc014066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:12.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc014098e30 0x7fc01409c7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:12.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 --2- 172.21.15.117:0/1115563018 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc01409b700 0x7fc0141459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:12.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 >> 172.21.15.117:0/1115563018 conn(0x7fc01405b9f0 msgr2=0x7fc0140b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:12.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 shutdown_connections 2020-11-20T13:56:12.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:12.147+0000 7fc01912c700 1 -- 172.21.15.117:0/1115563018 wait complete. 2020-11-20T13:56:12.151 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2020-11-20T13:56:12.361 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:55:50.675265+0000', 'modified': '2020-11-20T13:55:51.676934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/1587825839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1587825839}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1587825839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:12.361 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:12.362 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:12.490 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":4288,"want_state":"up:active","state":"up:active","rank_uptime":21.804853678000001,"mdsmap_epoch":5,"osdmap_epoch":25,"osdmap_epoch_barrier":25,"uptime":28.194779886999999} 2020-11-20T13:56:12.490 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 4288, "mdsmap_epoch": 5, "osdmap_epoch": 25, "osdmap_epoch_barrier": 25, "rank_uptime": 21.804853678, "state": "up:active", "uptime": 28.194779887, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:12.491 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:12.491 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2020-11-20T13:56:12.491 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.3 at ubuntu@smithi085.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3... 2020-11-20T13:56:12.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:12.492 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:12.511 INFO:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2020-11-20T13:56:12.582 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:12.583 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:12.651 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:12.653 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:12.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:12.654 INFO:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2020-11-20T13:56:12.710 INFO:teuthology.orchestra.run.smithi085.stdout:47 2020-11-20T13:56:12.711 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2020-11-20T13:56:12.711 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.3 /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:12.753 INFO:teuthology.orchestra.run.smithi085:> sudo modprobe fuse 2020-11-20T13:56:12.782 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:12.782 INFO:teuthology.orchestra.run.smithi085:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:12.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi085.stderr:2020-11-20T13:56:12.828+0000 7f0e0fcf51c0 -1 init, newargv = 0x5561874aecb0 newargc=9 2020-11-20T13:56:12.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi085.stderr:ceph-fuse[24037]: starting ceph client 2020-11-20T13:56:12.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi085.stderr:ceph-fuse[24037]: starting fuse 2020-11-20T13:56:12.851 INFO:teuthology.orchestra.run.smithi085.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:12.852 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:12.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:12.853 INFO:teuthology.orchestra.run.smithi085:> ls /sys/fs/fuse/connections 2020-11-20T13:56:12.906 INFO:teuthology.orchestra.run.smithi085.stdout:47 2020-11-20T13:56:12.906 INFO:teuthology.orchestra.run.smithi085.stdout:48 2020-11-20T13:56:12.907 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2020-11-20T13:56:12.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:12.907 INFO:teuthology.orchestra.run.smithi085:> sudo python3 -c ' 2020-11-20T13:56:12.908 INFO:teuthology.orchestra.run.smithi085:> import glob 2020-11-20T13:56:12.908 INFO:teuthology.orchestra.run.smithi085:> import re 2020-11-20T13:56:12.908 INFO:teuthology.orchestra.run.smithi085:> import os 2020-11-20T13:56:12.908 INFO:teuthology.orchestra.run.smithi085:> import subprocess 2020-11-20T13:56:12.909 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:12.909 INFO:teuthology.orchestra.run.smithi085:> def find_socket(client_name): 2020-11-20T13:56:12.909 INFO:teuthology.orchestra.run.smithi085:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2020-11-20T13:56:12.910 INFO:teuthology.orchestra.run.smithi085:> files = glob.glob(asok_path) 2020-11-20T13:56:12.910 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:12.910 INFO:teuthology.orchestra.run.smithi085:> # Given a non-glob path, it better be there 2020-11-20T13:56:12.910 INFO:teuthology.orchestra.run.smithi085:> if "*" not in asok_path: 2020-11-20T13:56:12.911 INFO:teuthology.orchestra.run.smithi085:> assert(len(files) == 1) 2020-11-20T13:56:12.911 INFO:teuthology.orchestra.run.smithi085:> return files[0] 2020-11-20T13:56:12.912 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:12.912 INFO:teuthology.orchestra.run.smithi085:> for f in files: 2020-11-20T13:56:12.912 INFO:teuthology.orchestra.run.smithi085:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:12.912 INFO:teuthology.orchestra.run.smithi085:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:12.913 INFO:teuthology.orchestra.run.smithi085:> return f 2020-11-20T13:56:12.913 INFO:teuthology.orchestra.run.smithi085:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:12.913 INFO:teuthology.orchestra.run.smithi085:> 2020-11-20T13:56:12.914 INFO:teuthology.orchestra.run.smithi085:> print(find_socket("client.3")) 2020-11-20T13:56:12.914 INFO:teuthology.orchestra.run.smithi085:> ' 2020-11-20T13:56:12.997 INFO:teuthology.orchestra.run.smithi085.stdout:/var/run/ceph/ceph-client.3.24037.asok 2020-11-20T13:56:13.001 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.24037.asok 2020-11-20T13:56:13.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.001 INFO:teuthology.orchestra.run.smithi085:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.24037.asok status 2020-11-20T13:56:13.104 INFO:teuthology.orchestra.run.smithi085.stdout:{ 2020-11-20T13:56:13.104 INFO:teuthology.orchestra.run.smithi085.stdout: "metadata": { 2020-11-20T13:56:13.105 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:13.105 INFO:teuthology.orchestra.run.smithi085.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:13.105 INFO:teuthology.orchestra.run.smithi085.stdout: "entity_id": "3", 2020-11-20T13:56:13.105 INFO:teuthology.orchestra.run.smithi085.stdout: "hostname": "smithi085", 2020-11-20T13:56:13.105 INFO:teuthology.orchestra.run.smithi085.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2020-11-20T13:56:13.106 INFO:teuthology.orchestra.run.smithi085.stdout: "pid": "24037", 2020-11-20T13:56:13.106 INFO:teuthology.orchestra.run.smithi085.stdout: "root": "/" 2020-11-20T13:56:13.106 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:13.106 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_count": 0, 2020-11-20T13:56:13.106 INFO:teuthology.orchestra.run.smithi085.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:13.107 INFO:teuthology.orchestra.run.smithi085.stdout: "id": 4653, 2020-11-20T13:56:13.107 INFO:teuthology.orchestra.run.smithi085.stdout: "inst": { 2020-11-20T13:56:13.107 INFO:teuthology.orchestra.run.smithi085.stdout: "name": { 2020-11-20T13:56:13.108 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "client", 2020-11-20T13:56:13.108 INFO:teuthology.orchestra.run.smithi085.stdout: "num": 4653 2020-11-20T13:56:13.108 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:13.108 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2020-11-20T13:56:13.109 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2020-11-20T13:56:13.109 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2020-11-20T13:56:13.109 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3057850748 2020-11-20T13:56:13.109 INFO:teuthology.orchestra.run.smithi085.stdout: } 2020-11-20T13:56:13.111 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:13.112 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": { 2020-11-20T13:56:13.112 INFO:teuthology.orchestra.run.smithi085.stdout: "type": "v1", 2020-11-20T13:56:13.112 INFO:teuthology.orchestra.run.smithi085.stdout: "addr": "172.21.15.85:0", 2020-11-20T13:56:13.112 INFO:teuthology.orchestra.run.smithi085.stdout: "nonce": 3057850748 2020-11-20T13:56:13.112 INFO:teuthology.orchestra.run.smithi085.stdout: }, 2020-11-20T13:56:13.113 INFO:teuthology.orchestra.run.smithi085.stdout: "inst_str": "client.4653 172.21.15.85:0/3057850748", 2020-11-20T13:56:13.113 INFO:teuthology.orchestra.run.smithi085.stdout: "addr_str": "172.21.15.85:0/3057850748", 2020-11-20T13:56:13.113 INFO:teuthology.orchestra.run.smithi085.stdout: "inode_count": 1, 2020-11-20T13:56:13.113 INFO:teuthology.orchestra.run.smithi085.stdout: "mds_epoch": 5, 2020-11-20T13:56:13.114 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch": 25, 2020-11-20T13:56:13.114 INFO:teuthology.orchestra.run.smithi085.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:13.114 INFO:teuthology.orchestra.run.smithi085.stdout: "blacklisted": false 2020-11-20T13:56:13.114 INFO:teuthology.orchestra.run.smithi085.stdout:} 2020-11-20T13:56:13.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.116 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:13.116 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:13.132 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:13.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:13.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.134 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:13.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.204 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:13.205 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:13.261 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:13.261 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-20T13:56:13.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.262 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:13.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.345 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:13.345 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:13.368 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2020-11-20T13:56:13.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-11-20T13:56:13.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.369 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:13.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.440 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:13.441 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:13.495 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2020-11-20T13:56:13.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2020-11-20T13:56:13.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:13.496 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:13.568 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-11-20T13:56:13.609 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:13.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.719+0000 7fef98123700 1 Processor -- start 2020-11-20T13:56:13.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.720+0000 7fef98123700 1 -- start start 2020-11-20T13:56:13.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.720+0000 7fef98123700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef90098e30 0x7fef9009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:13.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.720+0000 7fef98123700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900b5510 con 0x7fef900ae6f0 2020-11-20T13:56:13.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.720+0000 7fef98123700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900b57b0 con 0x7fef9009b700 2020-11-20T13:56:13.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.720+0000 7fef98123700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900b5a50 con 0x7fef90098e30 2020-11-20T13:56:13.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef95ebf700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef90098e30 0x7fef9009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:13.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef956be700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fef9009b700 0x7fef901438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60146/0 (socket says 172.21.15.117:60146) 2020-11-20T13:56:13.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef956be700 1 -- 172.21.15.117:0/2287432749 learned_addr learned my addr 172.21.15.117:0/2287432749 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:13.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef95ebf700 1 -- 172.21.15.117:0/2287432749 >> v1:172.21.15.117:6789/0 conn(0x7fef9009b700 legacy=0x7fef901438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:13.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef95ebf700 1 -- 172.21.15.117:0/2287432749 >> v1:172.21.15.85:6789/0 conn(0x7fef900ae6f0 legacy=0x7fef900aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:13.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef94ebd700 1 -- 172.21.15.117:0/2287432749 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103393812 0 0) 0x7fef900b57b0 con 0x7fef9009b700 2020-11-20T13:56:13.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef95ebf700 1 -- 172.21.15.117:0/2287432749 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90099210 con 0x7fef90098e30 2020-11-20T13:56:13.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef95ebf700 1 --2- 172.21.15.117:0/2287432749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef90098e30 0x7fef9009b210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fef90060b70 tx=0x7fef80006dc0).ready entity=mon.1 client_cookie=b09740e8f1f2c0ae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:13.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef94ebd700 1 -- 172.21.15.117:0/2287432749 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef800097b0 con 0x7fef90098e30 2020-11-20T13:56:13.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef94ebd700 1 -- 172.21.15.117:0/2287432749 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80009910 con 0x7fef90098e30 2020-11-20T13:56:13.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef94ebd700 1 -- 172.21.15.117:0/2287432749 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef80009c70 con 0x7fef90098e30 2020-11-20T13:56:13.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef98123700 1 -- 172.21.15.117:0/2287432749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef90098e30 msgr2=0x7fef9009b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:13.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.721+0000 7fef98123700 1 --2- 172.21.15.117:0/2287432749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef90098e30 0x7fef9009b210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fef90060b70 tx=0x7fef80006dc0).stop 2020-11-20T13:56:13.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.722+0000 7fef98123700 1 -- 172.21.15.117:0/2287432749 shutdown_connections 2020-11-20T13:56:13.727 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.722+0000 7fef98123700 1 --2- 172.21.15.117:0/2287432749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef90098e30 0x7fef9009b210 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:13.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.722+0000 7fef98123700 1 -- 172.21.15.117:0/2287432749 >> 172.21.15.117:0/2287432749 conn(0x7fef9005b9f0 msgr2=0x7fef9005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:13.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.722+0000 7fef98123700 1 -- 172.21.15.117:0/2287432749 shutdown_connections 2020-11-20T13:56:13.729 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.722+0000 7fef98123700 1 -- 172.21.15.117:0/2287432749 wait complete. 2020-11-20T13:56:13.732 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 Processor -- start 2020-11-20T13:56:13.732 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 -- start start 2020-11-20T13:56:13.732 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fef90098e30 0x7fef9013d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:13.732 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fef9009b700 0x7fef9013f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:13.733 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef900ae6f0 0x7fef900afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:13.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9013f710 con 0x7fef9009b700 2020-11-20T13:56:13.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9013e4d0 con 0x7fef900ae6f0 2020-11-20T13:56:13.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef98123700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9013e840 con 0x7fef90098e30 2020-11-20T13:56:13.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef966c0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef900ae6f0 0x7fef900afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:13.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef966c0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef900ae6f0 0x7fef900afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40994/0 (socket says 172.21.15.117:40994) 2020-11-20T13:56:13.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef966c0700 1 -- 172.21.15.117:0/2186950782 learned_addr learned my addr 172.21.15.117:0/2186950782 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:13.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.723+0000 7fef956be700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fef9009b700 0x7fef9013f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:13.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef95ebf700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fef90098e30 0x7fef9013d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:13.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef966c0700 1 -- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fef90098e30 msgr2=0x7fef9013d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:13.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef966c0700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fef90098e30 0x7fef9013d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:13.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef966c0700 1 -- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fef9009b700 msgr2=0x7fef9013f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:13.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef966c0700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fef9009b700 0x7fef9013f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:13.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef966c0700 1 -- 172.21.15.117:0/2186950782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9013eae0 con 0x7fef900ae6f0 2020-11-20T13:56:13.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef966c0700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef900ae6f0 0x7fef900afd60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fef880012a0 tx=0x7fef88007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:13.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef7effd700 1 -- 172.21.15.117:0/2186950782 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef8800a5a0 con 0x7fef900ae6f0 2020-11-20T13:56:13.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef7effd700 1 -- 172.21.15.117:0/2186950782 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800a700 con 0x7fef900ae6f0 2020-11-20T13:56:13.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef7effd700 1 -- 172.21.15.117:0/2186950782 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef8800aa60 con 0x7fef900ae6f0 2020-11-20T13:56:13.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9013ed80 con 0x7fef900ae6f0 2020-11-20T13:56:13.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.724+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef90146240 con 0x7fef900ae6f0 2020-11-20T13:56:13.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.725+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9013ef10 con 0x7fef900ae6f0 2020-11-20T13:56:13.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.725+0000 7fef7effd700 1 -- 172.21.15.117:0/2186950782 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fef88019030 con 0x7fef900ae6f0 2020-11-20T13:56:13.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.725+0000 7fef7effd700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef70051210 0x7fef700536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:13.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.725+0000 7fef7effd700 1 -- 172.21.15.117:0/2186950782 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fef88068940 con 0x7fef900ae6f0 2020-11-20T13:56:13.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.725+0000 7fef95ebf700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef70051210 0x7fef700536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:13.739 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.726+0000 7fef95ebf700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef70051210 0x7fef700536d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fef8001b000 tx=0x7fef80015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:13.739 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.729+0000 7fef7effd700 1 -- 172.21.15.117:0/2186950782 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fef88040c10 con 0x7fef900ae6f0 2020-11-20T13:56:13.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.878+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef90057050 con 0x7fef900ae6f0 2020-11-20T13:56:13.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.879+0000 7fef7effd700 1 -- 172.21.15.117:0/2186950782 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fef88024ff0 con 0x7fef900ae6f0 2020-11-20T13:56:13.881 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:13.881 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:13.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef70051210 msgr2=0x7fef700536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:13.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef70051210 0x7fef700536d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fef8001b000 tx=0x7fef80015000).stop 2020-11-20T13:56:13.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef900ae6f0 msgr2=0x7fef900afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:13.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef900ae6f0 0x7fef900afd60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fef880012a0 tx=0x7fef88007ef0).stop 2020-11-20T13:56:13.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 shutdown_connections 2020-11-20T13:56:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fef70051210 0x7fef700536d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fef90098e30 0x7fef9013d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef900ae6f0 0x7fef900afd60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 --2- 172.21.15.117:0/2186950782 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fef9009b700 0x7fef9013f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:13.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 >> 172.21.15.117:0/2186950782 conn(0x7fef9005b9f0 msgr2=0x7fef900b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 shutdown_connections 2020-11-20T13:56:13.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:13.881+0000 7fef98123700 1 -- 172.21.15.117:0/2186950782 wait complete. 2020-11-20T13:56:13.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:13.899 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:14.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.007+0000 7fb7f8908700 1 Processor -- start 2020-11-20T13:56:14.008 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.007+0000 7fb7f8908700 1 -- start start 2020-11-20T13:56:14.011 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.007+0000 7fb7f8908700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb7f409cb40 0x7fb7f41437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.011 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.007+0000 7fb7f8908700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f4143c90 0x7fb7f40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.011 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.007+0000 7fb7f8908700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f40999d0 con 0x7fb7f409bd80 2020-11-20T13:56:14.012 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f8908700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f40997f0 con 0x7fb7f409cb40 2020-11-20T13:56:14.012 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f8908700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4099c70 con 0x7fb7f4143c90 2020-11-20T13:56:14.012 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f259c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb7f409bd80 0x7fb7f409a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60158/0 (socket says 172.21.15.117:60158) 2020-11-20T13:56:14.012 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f259c700 1 -- 172.21.15.117:0/2411668159 learned_addr learned my addr 172.21.15.117:0/2411668159 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:14.013 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f2d9d700 1 --2- 172.21.15.117:0/2411668159 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f4143c90 0x7fb7f40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f1d9b700 1 --2- 172.21.15.117:0/2411668159 >> v2:172.21.15.85:3300/0 conn(0x7fb7f409cb40 0x7fb7f41437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f159a700 1 -- 172.21.15.117:0/2411668159 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189664499 0 0) 0x7fb7f40999d0 con 0x7fb7f409bd80 2020-11-20T13:56:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f159a700 1 -- 172.21.15.117:0/2411668159 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d8003230 con 0x7fb7f409bd80 2020-11-20T13:56:14.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f2d9d700 1 -- 172.21.15.117:0/2411668159 >> v2:172.21.15.85:3300/0 conn(0x7fb7f409cb40 msgr2=0x7fb7f41437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f2d9d700 1 --2- 172.21.15.117:0/2411668159 >> v2:172.21.15.85:3300/0 conn(0x7fb7f409cb40 0x7fb7f41437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f2d9d700 1 -- 172.21.15.117:0/2411668159 >> v1:172.21.15.117:6789/0 conn(0x7fb7f409bd80 legacy=0x7fb7f409a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f2d9d700 1 -- 172.21.15.117:0/2411668159 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4144120 con 0x7fb7f4143c90 2020-11-20T13:56:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f1d9b700 1 --2- 172.21.15.117:0/2411668159 >> v2:172.21.15.85:3300/0 conn(0x7fb7f409cb40 0x7fb7f41437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:14.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f2d9d700 1 --2- 172.21.15.117:0/2411668159 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f4143c90 0x7fb7f40b4110 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb7ec0032b0 tx=0x7fb7ec0091a0).ready entity=mon.2 client_cookie=2c19175b9635437b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f159a700 1 -- 172.21.15.117:0/2411668159 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7ec00b940 con 0x7fb7f4143c90 2020-11-20T13:56:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f159a700 1 -- 172.21.15.117:0/2411668159 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec00baa0 con 0x7fb7f4143c90 2020-11-20T13:56:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f159a700 1 -- 172.21.15.117:0/2411668159 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7ec00bc10 con 0x7fb7f4143c90 2020-11-20T13:56:14.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f8908700 1 -- 172.21.15.117:0/2411668159 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f4143c90 msgr2=0x7fb7f40b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.008+0000 7fb7f8908700 1 --2- 172.21.15.117:0/2411668159 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f4143c90 0x7fb7f40b4110 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fb7ec0032b0 tx=0x7fb7ec0091a0).stop 2020-11-20T13:56:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.009+0000 7fb7f8908700 1 -- 172.21.15.117:0/2411668159 shutdown_connections 2020-11-20T13:56:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.009+0000 7fb7f8908700 1 --2- 172.21.15.117:0/2411668159 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f4143c90 0x7fb7f40b4110 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.009+0000 7fb7f8908700 1 --2- 172.21.15.117:0/2411668159 >> v2:172.21.15.85:3300/0 conn(0x7fb7f409cb40 0x7fb7f41437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.009+0000 7fb7f8908700 1 -- 172.21.15.117:0/2411668159 >> 172.21.15.117:0/2411668159 conn(0x7fb7f40b9770 msgr2=0x7fb7f40b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:14.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.009+0000 7fb7f8908700 1 -- 172.21.15.117:0/2411668159 shutdown_connections 2020-11-20T13:56:14.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.009+0000 7fb7f8908700 1 -- 172.21.15.117:0/2411668159 wait complete. 2020-11-20T13:56:14.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 Processor -- start 2020-11-20T13:56:14.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 -- start start 2020-11-20T13:56:14.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb7f409bd80 0x7fb7f4067450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f409cb40 0x7fb7f4137b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f4143c90 0x7fb7f4157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4138030 con 0x7fb7f409bd80 2020-11-20T13:56:14.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f409e210 con 0x7fb7f4143c90 2020-11-20T13:56:14.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.010+0000 7fb7f8908700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f409e4b0 con 0x7fb7f409cb40 2020-11-20T13:56:14.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f4143c90 0x7fb7f4157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f1d9b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f409cb40 0x7fb7f4137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f4143c90 0x7fb7f4157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41008/0 (socket says 172.21.15.117:41008) 2020-11-20T13:56:14.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f259c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb7f409bd80 0x7fb7f4067450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 -- 172.21.15.117:0/3715625848 learned_addr learned my addr 172.21.15.117:0/3715625848 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:14.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 -- 172.21.15.117:0/3715625848 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f409cb40 msgr2=0x7fb7f4137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f409cb40 0x7fb7f4137b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 -- 172.21.15.117:0/3715625848 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb7f409bd80 msgr2=0x7fb7f4067450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb7f409bd80 0x7fb7f4067450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 -- 172.21.15.117:0/3715625848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f409e750 con 0x7fb7f4143c90 2020-11-20T13:56:14.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f2d9d700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f4143c90 0x7fb7f4157ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb7ec0032b0 tx=0x7fb7ec016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7e37fe700 1 -- 172.21.15.117:0/3715625848 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7ec0205e0 con 0x7fb7f4143c90 2020-11-20T13:56:14.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7e37fe700 1 -- 172.21.15.117:0/3715625848 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec020740 con 0x7fb7f4143c90 2020-11-20T13:56:14.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f40ae6d0 con 0x7fb7f4143c90 2020-11-20T13:56:14.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7e37fe700 1 -- 172.21.15.117:0/3715625848 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7ec020aa0 con 0x7fb7f4143c90 2020-11-20T13:56:14.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.011+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f4157fd0 con 0x7fb7f4143c90 2020-11-20T13:56:14.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.012+0000 7fb7e37fe700 1 -- 172.21.15.117:0/3715625848 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb7ec01e070 con 0x7fb7f4143c90 2020-11-20T13:56:14.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.012+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f4057050 con 0x7fb7f4143c90 2020-11-20T13:56:14.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.012+0000 7fb7e37fe700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb7d0051210 0x7fb7d00536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.013+0000 7fb7e37fe700 1 -- 172.21.15.117:0/3715625848 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fb7ec069c10 con 0x7fb7f4143c90 2020-11-20T13:56:14.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.013+0000 7fb7f259c700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb7d0051210 0x7fb7d00536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.013+0000 7fb7f259c700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb7d0051210 0x7fb7d00536d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fb7e800d000 tx=0x7fb7e8007970).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.016+0000 7fb7e37fe700 1 -- 172.21.15.117:0/3715625848 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb7ec041e30 con 0x7fb7f4143c90 2020-11-20T13:56:14.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.165+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb7f405ca00 con 0x7fb7f4143c90 2020-11-20T13:56:14.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.165+0000 7fb7e37fe700 1 -- 172.21.15.117:0/3715625848 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb7ec015020 con 0x7fb7f4143c90 2020-11-20T13:56:14.167 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:14.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb7d0051210 msgr2=0x7fb7d00536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb7d0051210 0x7fb7d00536d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fb7e800d000 tx=0x7fb7e8007970).stop 2020-11-20T13:56:14.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f4143c90 msgr2=0x7fb7f4157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f4143c90 0x7fb7f4157ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb7ec0032b0 tx=0x7fb7ec016000).stop 2020-11-20T13:56:14.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 shutdown_connections 2020-11-20T13:56:14.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb7d0051210 0x7fb7d00536d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7f4143c90 0x7fb7f4157ae0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb7f409bd80 0x7fb7f4067450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 --2- 172.21.15.117:0/3715625848 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb7f409cb40 0x7fb7f4137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 >> 172.21.15.117:0/3715625848 conn(0x7fb7f40b9770 msgr2=0x7fb7f4139090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:14.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 shutdown_connections 2020-11-20T13:56:14.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.167+0000 7fb7f8908700 1 -- 172.21.15.117:0/3715625848 wait complete. 2020-11-20T13:56:14.178 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:14.288 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.287+0000 7f01722e1700 1 Processor -- start 2020-11-20T13:56:14.289 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.287+0000 7f01722e1700 1 -- start start 2020-11-20T13:56:14.291 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f01722e1700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c05fb90 0x7f016c1434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.291 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f01722e1700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c144030 con 0x7f016c0b5a60 2020-11-20T13:56:14.291 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f01722e1700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c1442d0 con 0x7f016c099270 2020-11-20T13:56:14.291 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f01722e1700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c143e20 con 0x7f016c05fb90 2020-11-20T13:56:14.292 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f0171ae0700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f016c099270 0x7f016c099700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60174/0 (socket says 172.21.15.117:60174) 2020-11-20T13:56:14.292 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f0171ae0700 1 -- 172.21.15.117:0/618398579 learned_addr learned my addr 172.21.15.117:0/618398579 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:14.292 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f01712df700 1 --2- 172.21.15.117:0/618398579 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c05fb90 0x7f016c1434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.292 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.288+0000 7f015bfff700 1 -- 172.21.15.117:0/618398579 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110126511 0 0) 0x7f016c1442d0 con 0x7f016c099270 2020-11-20T13:56:14.292 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f015bfff700 1 -- 172.21.15.117:0/618398579 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0154004340 con 0x7f016c099270 2020-11-20T13:56:14.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f01712df700 1 -- 172.21.15.117:0/618398579 >> v1:172.21.15.117:6789/0 conn(0x7f016c099270 legacy=0x7f016c099700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f01712df700 1 -- 172.21.15.117:0/618398579 >> v1:172.21.15.85:6789/0 conn(0x7f016c0b5a60 legacy=0x7f016c13d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f01712df700 1 -- 172.21.15.117:0/618398579 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c05ff70 con 0x7f016c05fb90 2020-11-20T13:56:14.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f01712df700 1 --2- 172.21.15.117:0/618398579 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c05fb90 0x7f016c1434b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f015c002f30 tx=0x7f015c006dc0).ready entity=mon.2 client_cookie=62bf8ca210573a5d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f015bfff700 1 -- 172.21.15.117:0/618398579 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c00fa10 con 0x7f016c05fb90 2020-11-20T13:56:14.294 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f015bfff700 1 -- 172.21.15.117:0/618398579 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c00fb70 con 0x7f016c05fb90 2020-11-20T13:56:14.294 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f015bfff700 1 -- 172.21.15.117:0/618398579 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f015c010470 con 0x7f016c05fb90 2020-11-20T13:56:14.294 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f01722e1700 1 -- 172.21.15.117:0/618398579 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c05fb90 msgr2=0x7f016c1434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.294 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.289+0000 7f01722e1700 1 --2- 172.21.15.117:0/618398579 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c05fb90 0x7f016c1434b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f015c002f30 tx=0x7f015c006dc0).stop 2020-11-20T13:56:14.294 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.290+0000 7f01722e1700 1 -- 172.21.15.117:0/618398579 shutdown_connections 2020-11-20T13:56:14.295 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.290+0000 7f01722e1700 1 --2- 172.21.15.117:0/618398579 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c05fb90 0x7f016c1434b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.295 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.290+0000 7f01722e1700 1 -- 172.21.15.117:0/618398579 >> 172.21.15.117:0/618398579 conn(0x7f016c0b9490 msgr2=0x7f016c0b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:14.295 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.290+0000 7f01722e1700 1 -- 172.21.15.117:0/618398579 shutdown_connections 2020-11-20T13:56:14.295 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.290+0000 7f01722e1700 1 -- 172.21.15.117:0/618398579 wait complete. 2020-11-20T13:56:14.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 Processor -- start 2020-11-20T13:56:14.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 -- start start 2020-11-20T13:56:14.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f016c05fb90 0x7f016c144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c099270 0x7f016c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c0b5a60 0x7f016c066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c1445e0 con 0x7f016c05fb90 2020-11-20T13:56:14.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c144880 con 0x7f016c0b5a60 2020-11-20T13:56:14.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01722e1700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c144b20 con 0x7f016c099270 2020-11-20T13:56:14.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f0171ae0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c0b5a60 0x7f016c066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f0171ae0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c0b5a60 0x7f016c066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41022/0 (socket says 172.21.15.117:41022) 2020-11-20T13:56:14.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f0171ae0700 1 -- 172.21.15.117:0/3811011212 learned_addr learned my addr 172.21.15.117:0/3811011212 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:14.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f01712df700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f016c05fb90 0x7f016c144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.291+0000 7f0170ade700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c099270 0x7f016c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f0171ae0700 1 -- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c099270 msgr2=0x7f016c1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f0171ae0700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c099270 0x7f016c1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f0171ae0700 1 -- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f016c05fb90 msgr2=0x7f016c144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f0171ae0700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f016c05fb90 0x7f016c144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f0171ae0700 1 -- 172.21.15.117:0/3811011212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c156ed0 con 0x7f016c0b5a60 2020-11-20T13:56:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f0171ae0700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c0b5a60 0x7f016c066840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f0164001450 tx=0x7f0164007d30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f015a7fc700 1 -- 172.21.15.117:0/3811011212 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f016400ce10 con 0x7f016c0b5a60 2020-11-20T13:56:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f015a7fc700 1 -- 172.21.15.117:0/3811011212 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164005b70 con 0x7f016c0b5a60 2020-11-20T13:56:14.313 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c13e650 con 0x7f016c0b5a60 2020-11-20T13:56:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f015a7fc700 1 -- 172.21.15.117:0/3811011212 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0164007110 con 0x7f016c0b5a60 2020-11-20T13:56:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.292+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c066f70 con 0x7f016c0b5a60 2020-11-20T13:56:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.293+0000 7f015a7fc700 1 -- 172.21.15.117:0/3811011212 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f0164010030 con 0x7f016c0b5a60 2020-11-20T13:56:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.293+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c144ce0 con 0x7f016c0b5a60 2020-11-20T13:56:14.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.293+0000 7f015a7fc700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f014c051210 0x7f014c0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.293+0000 7f015a7fc700 1 -- 172.21.15.117:0/3811011212 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f0164068ff0 con 0x7f016c0b5a60 2020-11-20T13:56:14.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.293+0000 7f01712df700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f014c051210 0x7f014c0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.294+0000 7f01712df700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f014c051210 0x7f014c0536d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f015c01a000 tx=0x7f015c017000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.297+0000 7f015a7fc700 1 -- 172.21.15.117:0/3811011212 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f0164041210 con 0x7f016c0b5a60 2020-11-20T13:56:14.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.450+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f016c057050 con 0x7f016c0b5a60 2020-11-20T13:56:14.451 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.450+0000 7f015a7fc700 1 -- 172.21.15.117:0/3811011212 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0164044830 con 0x7f016c0b5a60 2020-11-20T13:56:14.452 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:14.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f014c051210 msgr2=0x7f014c0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.453 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f014c051210 0x7f014c0536d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f015c01a000 tx=0x7f015c017000).stop 2020-11-20T13:56:14.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c0b5a60 msgr2=0x7f016c066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c0b5a60 0x7f016c066840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f0164001450 tx=0x7f0164007d30).stop 2020-11-20T13:56:14.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 shutdown_connections 2020-11-20T13:56:14.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f014c051210 0x7f014c0536d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f016c0b5a60 0x7f016c066840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f016c05fb90 0x7f016c144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 --2- 172.21.15.117:0/3811011212 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f016c099270 0x7f016c1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 >> 172.21.15.117:0/3811011212 conn(0x7f016c0b9490 msgr2=0x7f016c0b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:14.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 shutdown_connections 2020-11-20T13:56:14.455 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.452+0000 7f01722e1700 1 -- 172.21.15.117:0/3811011212 wait complete. 2020-11-20T13:56:14.463 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:14.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.570+0000 7f3b0838c700 1 Processor -- start 2020-11-20T13:56:14.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.570+0000 7f3b0838c700 1 -- start start 2020-11-20T13:56:14.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b0838c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 0x7f3b00147d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b0838c700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b00144ce0 con 0x7f3b0009f710 2020-11-20T13:56:14.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b0838c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0005b9f0 con 0x7f3b000af070 2020-11-20T13:56:14.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b0838c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000b4900 con 0x7f3b000b5b90 2020-11-20T13:56:14.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b06929700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3b000af070 0x7f3b000af500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60186/0 (socket says 172.21.15.117:60186) 2020-11-20T13:56:14.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b06929700 1 -- 172.21.15.117:0/413287920 learned_addr learned my addr 172.21.15.117:0/413287920 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:14.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b05927700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 0x7f3b00147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.571+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708493020 0 0) 0x7f3b0005b9f0 con 0x7f3b000af070 2020-11-20T13:56:14.585 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae8003230 con 0x7f3b000af070 2020-11-20T13:56:14.585 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3193205729 0 0) 0x7f3ae8003230 con 0x7f3b000af070 2020-11-20T13:56:14.585 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 msgr2=0x7f3b00147d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 0x7f3b00147d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 >> v1:172.21.15.85:6789/0 conn(0x7f3b0009f710 legacy=0x7f3b000b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000b6180 con 0x7f3b000af070 2020-11-20T13:56:14.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f3af8001a50 con 0x7f3b000af070 2020-11-20T13:56:14.589 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 >> v1:172.21.15.117:6789/0 conn(0x7f3b000af070 legacy=0x7f3b000af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3ae8003750 0x7f3ae8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ae8004090 0x7f3ae8004570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 0x7f3ae8004bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000b4900 con 0x7f3ae8003750 2020-11-20T13:56:14.590 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8005110 con 0x7f3ae8004090 2020-11-20T13:56:14.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae80052d0 con 0x7f3b000b5b90 2020-11-20T13:56:14.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05927700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ae8004090 0x7f3ae8004570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b06128700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3ae8003750 0x7f3ae8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b06929700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 0x7f3ae8004bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05927700 1 -- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 msgr2=0x7f3ae8004bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05927700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 0x7f3ae8004bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05927700 1 -- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3ae8003750 msgr2=0x7f3ae8003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05927700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3ae8003750 0x7f3ae8003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05927700 1 -- 172.21.15.117:0/413287920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae8003230 con 0x7f3ae8004090 2020-11-20T13:56:14.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b06128700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3ae8003750 0x7f3ae8003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:14.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b06929700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b5b90 0x7f3ae8004bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:14.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05927700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ae8004090 0x7f3ae8004570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f3afc00c000 tx=0x7f3afc008040).ready entity=mon.1 client_cookie=c0800b6d2adf6a74 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3afc00e9d0 con 0x7f3ae8004090 2020-11-20T13:56:14.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00eb30 con 0x7f3ae8004090 2020-11-20T13:56:14.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b05126700 1 -- 172.21.15.117:0/413287920 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3afc00ee90 con 0x7f3ae8004090 2020-11-20T13:56:14.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b0838c700 1 -- 172.21.15.117:0/413287920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ae8004090 msgr2=0x7f3ae8004570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b0838c700 1 --2- 172.21.15.117:0/413287920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ae8004090 0x7f3ae8004570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f3afc00c000 tx=0x7f3afc008040).stop 2020-11-20T13:56:14.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.572+0000 7f3b06929700 1 -- 172.21.15.117:0/413287920 reap_dead start 2020-11-20T13:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.573+0000 7f3b0838c700 1 -- 172.21.15.117:0/413287920 shutdown_connections 2020-11-20T13:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.573+0000 7f3b0838c700 1 -- 172.21.15.117:0/413287920 >> 172.21.15.117:0/413287920 conn(0x7f3b000b6970 msgr2=0x7f3b000b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.573+0000 7f3b0838c700 1 -- 172.21.15.117:0/413287920 shutdown_connections 2020-11-20T13:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.573+0000 7f3b0838c700 1 -- 172.21.15.117:0/413287920 wait complete. 2020-11-20T13:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.574+0000 7f3b0838c700 1 Processor -- start 2020-11-20T13:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.574+0000 7f3b0838c700 1 -- start start 2020-11-20T13:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b001489f0 0x7f3b000b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b2a30 0x7f3b000b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b00142b20 0x7f3b00142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00148210 con 0x7f3b001489f0 2020-11-20T13:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001434a0 con 0x7f3b00142b20 2020-11-20T13:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000b2ef0 con 0x7f3b000b2a30 2020-11-20T13:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b00142b20 0x7f3b00142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b00142b20 0x7f3b00142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41040/0 (socket says 172.21.15.117:41040) 2020-11-20T13:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 -- 172.21.15.117:0/2947293768 learned_addr learned my addr 172.21.15.117:0/2947293768 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06128700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b001489f0 0x7f3b000b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b05927700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b2a30 0x7f3b000b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 -- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b2a30 msgr2=0x7f3b000b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b2a30 0x7f3b000b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 -- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b001489f0 msgr2=0x7f3b000b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b001489f0 0x7f3b000b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 -- 172.21.15.117:0/2947293768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000b3190 con 0x7f3b00142b20 2020-11-20T13:56:14.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b05927700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b2a30 0x7f3b000b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:14.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06128700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b001489f0 0x7f3b000b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:14.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b06929700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b00142b20 0x7f3b00142fb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f3b0009e8f0 tx=0x7f3af80012f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3aef7fe700 1 -- 172.21.15.117:0/2947293768 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af800ca70 con 0x7f3b00142b20 2020-11-20T13:56:14.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3aef7fe700 1 -- 172.21.15.117:0/2947293768 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af800cbd0 con 0x7f3b00142b20 2020-11-20T13:56:14.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b000b3430 con 0x7f3b00142b20 2020-11-20T13:56:14.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3aef7fe700 1 -- 172.21.15.117:0/2947293768 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af8014430 con 0x7f3b00142b20 2020-11-20T13:56:14.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.575+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b000b2770 con 0x7f3b00142b20 2020-11-20T13:56:14.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.576+0000 7f3aef7fe700 1 -- 172.21.15.117:0/2947293768 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3af8014590 con 0x7f3b00142b20 2020-11-20T13:56:14.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.576+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b000b35c0 con 0x7f3b00142b20 2020-11-20T13:56:14.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.577+0000 7f3aef7fe700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3ae0051290 0x7f3ae0053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.577+0000 7f3aef7fe700 1 -- 172.21.15.117:0/2947293768 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3af80690c0 con 0x7f3b00142b20 2020-11-20T13:56:14.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.577+0000 7f3b06128700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3ae0051290 0x7f3ae0053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.577+0000 7f3b06128700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3ae0051290 0x7f3ae0053750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3af000d000 tx=0x7f3af0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.580+0000 7f3aef7fe700 1 -- 172.21.15.117:0/2947293768 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3af8041170 con 0x7f3b00142b20 2020-11-20T13:56:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.733+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b00057050 con 0x7f3b00142b20 2020-11-20T13:56:14.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.733+0000 7f3aef7fe700 1 -- 172.21.15.117:0/2947293768 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3af8044790 con 0x7f3b00142b20 2020-11-20T13:56:14.735 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3ae0051290 msgr2=0x7f3ae0053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3ae0051290 0x7f3ae0053750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3af000d000 tx=0x7f3af0009040).stop 2020-11-20T13:56:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b00142b20 msgr2=0x7f3b00142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b00142b20 0x7f3b00142fb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f3b0009e8f0 tx=0x7f3af80012f0).stop 2020-11-20T13:56:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 shutdown_connections 2020-11-20T13:56:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3ae0051290 0x7f3ae0053750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b00142b20 0x7f3b00142fb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3b001489f0 0x7f3b000b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 --2- 172.21.15.117:0/2947293768 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3b000b2a30 0x7f3b000b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 >> 172.21.15.117:0/2947293768 conn(0x7f3b000b6970 msgr2=0x7f3b000af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 shutdown_connections 2020-11-20T13:56:14.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.735+0000 7f3b0838c700 1 -- 172.21.15.117:0/2947293768 wait complete. 2020-11-20T13:56:14.746 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:14.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.852+0000 7ff76e826700 1 Processor -- start 2020-11-20T13:56:14.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76e826700 1 -- start start 2020-11-20T13:56:14.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76e826700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 0x7ff768149420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76e826700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7ff768098dc0 0x7ff768145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76e826700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff768148a40 con 0x7ff7681447b0 2020-11-20T13:56:14.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76e826700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7681462f0 con 0x7ff768098dc0 2020-11-20T13:56:14.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76e826700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff768148ce0 con 0x7ff76809b930 2020-11-20T13:56:14.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76cdc3700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff7681447b0 0x7ff768144c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60208/0 (socket says 172.21.15.117:60208) 2020-11-20T13:56:14.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.853+0000 7ff76cdc3700 1 -- 172.21.15.117:0/3119319859 learned_addr learned my addr 172.21.15.117:0/3119319859 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:14.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff767fff700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 0x7ff768149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff7677fe700 1 --2- 172.21.15.117:0/3119319859 >> v2:172.21.15.85:3300/0 conn(0x7ff768098dc0 0x7ff768145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105475309 0 0) 0x7ff768148a40 con 0x7ff7681447b0 2020-11-20T13:56:14.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff754003230 con 0x7ff7681447b0 2020-11-20T13:56:14.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1739453836 0 0) 0x7ff754003230 con 0x7ff7681447b0 2020-11-20T13:56:14.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 msgr2=0x7ff768149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 0x7ff768149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 >> v2:172.21.15.85:3300/0 conn(0x7ff768098dc0 msgr2=0x7ff768145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 --2- 172.21.15.117:0/3119319859 >> v2:172.21.15.85:3300/0 conn(0x7ff768098dc0 0x7ff768145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768148f80 con 0x7ff7681447b0 2020-11-20T13:56:14.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7ff760001a50 con 0x7ff7681447b0 2020-11-20T13:56:14.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 >> v1:172.21.15.117:6789/0 conn(0x7ff7681447b0 legacy=0x7ff768144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff754003720 0x7ff754003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 0x7ff7540044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7540049c0 0x7ff754004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7681462f0 con 0x7ff754003720 2020-11-20T13:56:14.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff768148ce0 con 0x7ff7540049c0 2020-11-20T13:56:14.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7540041c0 con 0x7ff76809b930 2020-11-20T13:56:14.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff76cdc3700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7540049c0 0x7ff754004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff767fff700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff754003720 0x7ff754003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff7677fe700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 0x7ff7540044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff76cdc3700 1 -- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 msgr2=0x7ff7540044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff76cdc3700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff76809b930 0x7ff7540044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff76cdc3700 1 -- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff754003720 msgr2=0x7ff754003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:14.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff76cdc3700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff754003720 0x7ff754003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff76cdc3700 1 -- 172.21.15.117:0/3119319859 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754003230 con 0x7ff7540049c0 2020-11-20T13:56:14.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.854+0000 7ff76cdc3700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7540049c0 0x7ff754004e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7ff760001760 tx=0x7ff76809cca0).ready entity=mon.1 client_cookie=b60add7eae7ecba2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff760005b70 con 0x7ff7540049c0 2020-11-20T13:56:14.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff760005cd0 con 0x7ff7540049c0 2020-11-20T13:56:14.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff766ffd700 1 -- 172.21.15.117:0/3119319859 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff760007300 con 0x7ff7540049c0 2020-11-20T13:56:14.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff76e826700 1 -- 172.21.15.117:0/3119319859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7540049c0 msgr2=0x7ff754004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff76e826700 1 --2- 172.21.15.117:0/3119319859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7540049c0 0x7ff754004e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7ff760001760 tx=0x7ff76809cca0).stop 2020-11-20T13:56:14.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff76cdc3700 1 -- 172.21.15.117:0/3119319859 reap_dead start 2020-11-20T13:56:14.873 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff76e826700 1 -- 172.21.15.117:0/3119319859 shutdown_connections 2020-11-20T13:56:14.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff76e826700 1 -- 172.21.15.117:0/3119319859 >> 172.21.15.117:0/3119319859 conn(0x7ff7680b9770 msgr2=0x7ff7680b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:14.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff76e826700 1 -- 172.21.15.117:0/3119319859 shutdown_connections 2020-11-20T13:56:14.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.855+0000 7ff76e826700 1 -- 172.21.15.117:0/3119319859 wait complete. 2020-11-20T13:56:14.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.856+0000 7ff76e826700 1 Processor -- start 2020-11-20T13:56:14.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.856+0000 7ff76e826700 1 -- start start 2020-11-20T13:56:14.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76e826700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7680b2e60 0x7ff76809ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76e826700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff76808f770 0x7ff768159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76e826700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff76815a400 0x7ff7680993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76e826700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76809b150 con 0x7ff76808f770 2020-11-20T13:56:14.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76e826700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff768159950 con 0x7ff76815a400 2020-11-20T13:56:14.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76e826700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff768098120 con 0x7ff7680b2e60 2020-11-20T13:56:14.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff76815a400 0x7ff7680993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff76815a400 0x7ff7680993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41060/0 (socket says 172.21.15.117:41060) 2020-11-20T13:56:14.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 -- 172.21.15.117:0/46808178 learned_addr learned my addr 172.21.15.117:0/46808178 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:14.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff7677fe700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff76808f770 0x7ff768159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff767fff700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7680b2e60 0x7ff76809ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 -- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7680b2e60 msgr2=0x7ff76809ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7680b2e60 0x7ff76809ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 -- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff76808f770 msgr2=0x7ff768159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:14.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff76808f770 0x7ff768159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:14.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 -- 172.21.15.117:0/46808178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768098820 con 0x7ff76815a400 2020-11-20T13:56:14.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.857+0000 7ff76cdc3700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff76815a400 0x7ff7680993a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff76809cd10 tx=0x7ff760014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.858+0000 7ff7657fa700 1 -- 172.21.15.117:0/46808178 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff760007ce0 con 0x7ff76815a400 2020-11-20T13:56:14.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.858+0000 7ff7657fa700 1 -- 172.21.15.117:0/46808178 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff760013810 con 0x7ff76815a400 2020-11-20T13:56:14.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.858+0000 7ff7657fa700 1 -- 172.21.15.117:0/46808178 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff760001e70 con 0x7ff76815a400 2020-11-20T13:56:14.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.858+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff768098380 con 0x7ff76815a400 2020-11-20T13:56:14.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.858+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76809b370 con 0x7ff76815a400 2020-11-20T13:56:14.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.859+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff768098510 con 0x7ff76815a400 2020-11-20T13:56:14.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.859+0000 7ff7657fa700 1 -- 172.21.15.117:0/46808178 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff760025030 con 0x7ff76815a400 2020-11-20T13:56:14.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.859+0000 7ff7657fa700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff748051320 0x7ff7480537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:14.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.859+0000 7ff7657fa700 1 -- 172.21.15.117:0/46808178 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7ff76006a4e0 con 0x7ff76815a400 2020-11-20T13:56:14.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.859+0000 7ff767fff700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff748051320 0x7ff7480537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:14.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.859+0000 7ff767fff700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff748051320 0x7ff7480537e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7ff758008000 tx=0x7ff758008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:14.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:14.862+0000 7ff7657fa700 1 -- 172.21.15.117:0/46808178 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff760042700 con 0x7ff76815a400 2020-11-20T13:56:15.014 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.013+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff768057050 con 0x7ff76815a400 2020-11-20T13:56:15.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.013+0000 7ff7657fa700 1 -- 172.21.15.117:0/46808178 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff760023020 con 0x7ff76815a400 2020-11-20T13:56:15.015 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:15.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff748051320 msgr2=0x7ff7480537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff748051320 0x7ff7480537e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7ff758008000 tx=0x7ff758008040).stop 2020-11-20T13:56:15.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff76815a400 msgr2=0x7ff7680993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff76815a400 0x7ff7680993a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff76809cd10 tx=0x7ff760014000).stop 2020-11-20T13:56:15.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 shutdown_connections 2020-11-20T13:56:15.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff748051320 0x7ff7480537e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7680b2e60 0x7ff76809ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff76815a400 0x7ff7680993a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 --2- 172.21.15.117:0/46808178 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff76808f770 0x7ff768159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 >> 172.21.15.117:0/46808178 conn(0x7ff7680b9770 msgr2=0x7ff7680b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 shutdown_connections 2020-11-20T13:56:15.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.015+0000 7ff76e826700 1 -- 172.21.15.117:0/46808178 wait complete. 2020-11-20T13:56:15.026 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:15.133 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.131+0000 7fd36e2ad700 1 Processor -- start 2020-11-20T13:56:15.133 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.131+0000 7fd36e2ad700 1 -- start start 2020-11-20T13:56:15.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.132+0000 7fd36e2ad700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd3680af060 0x7fd3680af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.132+0000 7fd36e2ad700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3680ae450 con 0x7fd36805fb60 2020-11-20T13:56:15.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.132+0000 7fd36e2ad700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3680ae680 con 0x7fd368099e10 2020-11-20T13:56:15.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.132+0000 7fd36e2ad700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36809c100 con 0x7fd3680af060 2020-11-20T13:56:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.132+0000 7fd36caaa700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd368099e10 0x7fd3681430d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60224/0 (socket says 172.21.15.117:60224) 2020-11-20T13:56:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.132+0000 7fd36caaa700 1 -- 172.21.15.117:0/2640392490 learned_addr learned my addr 172.21.15.117:0/2640392490 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 --2- 172.21.15.117:0/2640392490 >> v2:172.21.15.85:3300/0 conn(0x7fd3680af060 0x7fd3680af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997415697 0 0) 0x7fd3680ae680 con 0x7fd368099e10 2020-11-20T13:56:15.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd350003230 con 0x7fd368099e10 2020-11-20T13:56:15.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158771709 0 0) 0x7fd3680ae450 con 0x7fd36805fb60 2020-11-20T13:56:15.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3680ae680 con 0x7fd36805fb60 2020-11-20T13:56:15.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 979901420 0 0) 0x7fd350003230 con 0x7fd368099e10 2020-11-20T13:56:15.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 >> v2:172.21.15.85:3300/0 conn(0x7fd3680af060 msgr2=0x7fd3680af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 --2- 172.21.15.117:0/2640392490 >> v2:172.21.15.85:3300/0 conn(0x7fd3680af060 0x7fd3680af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 >> v1:172.21.15.85:6789/0 conn(0x7fd36805fb60 legacy=0x7fd3681479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3680ae920 con 0x7fd368099e10 2020-11-20T13:56:15.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fd3640016d0 con 0x7fd368099e10 2020-11-20T13:56:15.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 >> v1:172.21.15.117:6789/0 conn(0x7fd368099e10 legacy=0x7fd3681430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3500038b0 0x7fd350003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3500044d0 0x7fd350004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd350004e40 0x7fd35000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35000bc10 con 0x7fd3500044d0 2020-11-20T13:56:15.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35000bd50 con 0x7fd350004e40 2020-11-20T13:56:15.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35000bee0 con 0x7fd3500038b0 2020-11-20T13:56:15.143 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd350004e40 0x7fd35000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.143 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36caaa700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3500044d0 0x7fd350004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.143 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36d2ab700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3500038b0 0x7fd350003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 -- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3500038b0 msgr2=0x7fd350003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3500038b0 0x7fd350003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 -- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3500044d0 msgr2=0x7fd350004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3500044d0 0x7fd350004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 -- 172.21.15.117:0/2640392490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd350003230 con 0x7fd350004e40 2020-11-20T13:56:15.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 -- 172.21.15.117:0/2640392490 reap_dead start 2020-11-20T13:56:15.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.133+0000 7fd36daac700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd350004e40 0x7fd35000b720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fd36000c000 tx=0x7fd3600091a0).ready entity=mon.1 client_cookie=871227ac558ed1cf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd36000ec30 con 0x7fd350004e40 2020-11-20T13:56:15.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36000ed90 con 0x7fd350004e40 2020-11-20T13:56:15.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd357fff700 1 -- 172.21.15.117:0/2640392490 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3600125e0 con 0x7fd350004e40 2020-11-20T13:56:15.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd36e2ad700 1 -- 172.21.15.117:0/2640392490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd350004e40 msgr2=0x7fd35000b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd350004e40 0x7fd35000b720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fd36000c000 tx=0x7fd3600091a0).stop 2020-11-20T13:56:15.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd36e2ad700 1 -- 172.21.15.117:0/2640392490 shutdown_connections 2020-11-20T13:56:15.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/2640392490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd350004e40 0x7fd35000b720 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd36e2ad700 1 -- 172.21.15.117:0/2640392490 >> 172.21.15.117:0/2640392490 conn(0x7fd3680b9470 msgr2=0x7fd3680b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd36e2ad700 1 -- 172.21.15.117:0/2640392490 shutdown_connections 2020-11-20T13:56:15.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.134+0000 7fd36e2ad700 1 -- 172.21.15.117:0/2640392490 wait complete. 2020-11-20T13:56:15.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.135+0000 7fd36e2ad700 1 Processor -- start 2020-11-20T13:56:15.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.135+0000 7fd36e2ad700 1 -- start start 2020-11-20T13:56:15.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36e2ad700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd350004e40 0x7fd368066b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36e2ad700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd36809e7b0 0x7fd36813d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36e2ad700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368139f50 0x7fd36813a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36e2ad700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36813dca0 con 0x7fd350004e40 2020-11-20T13:56:15.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36e2ad700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36809ecc0 con 0x7fd368139f50 2020-11-20T13:56:15.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36e2ad700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36809ef60 con 0x7fd36809e7b0 2020-11-20T13:56:15.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368139f50 0x7fd36813a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368139f50 0x7fd36813a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41076/0 (socket says 172.21.15.117:41076) 2020-11-20T13:56:15.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 -- 172.21.15.117:0/1829157825 learned_addr learned my addr 172.21.15.117:0/1829157825 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:15.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36d2ab700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd350004e40 0x7fd368066b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36caaa700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd36809e7b0 0x7fd36813d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 -- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd36809e7b0 msgr2=0x7fd36813d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd36809e7b0 0x7fd36813d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 -- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd350004e40 msgr2=0x7fd368066b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd350004e40 0x7fd368066b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 -- 172.21.15.117:0/1829157825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36809f200 con 0x7fd368139f50 2020-11-20T13:56:15.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.136+0000 7fd36daac700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368139f50 0x7fd36813a400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fd36000c000 tx=0x7fd360019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.137+0000 7fd3567fc700 1 -- 172.21.15.117:0/1829157825 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd36001b850 con 0x7fd368139f50 2020-11-20T13:56:15.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.137+0000 7fd3567fc700 1 -- 172.21.15.117:0/1829157825 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36001b9b0 con 0x7fd368139f50 2020-11-20T13:56:15.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.137+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36809c6f0 con 0x7fd368139f50 2020-11-20T13:56:15.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.137+0000 7fd3567fc700 1 -- 172.21.15.117:0/1829157825 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd36001bd10 con 0x7fd368139f50 2020-11-20T13:56:15.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.137+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36809cbb0 con 0x7fd368139f50 2020-11-20T13:56:15.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.138+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd368057050 con 0x7fd368139f50 2020-11-20T13:56:15.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.138+0000 7fd3567fc700 1 -- 172.21.15.117:0/1829157825 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd360018020 con 0x7fd368139f50 2020-11-20T13:56:15.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.138+0000 7fd3567fc700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd348051320 0x7fd3480537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.138+0000 7fd3567fc700 1 -- 172.21.15.117:0/1829157825 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fd36006b5b0 con 0x7fd368139f50 2020-11-20T13:56:15.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.138+0000 7fd36d2ab700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd348051320 0x7fd3480537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.138+0000 7fd36d2ab700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd348051320 0x7fd3480537e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fd35800d000 tx=0x7fd35800c040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.142+0000 7fd3567fc700 1 -- 172.21.15.117:0/1829157825 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd3600437d0 con 0x7fd368139f50 2020-11-20T13:56:15.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.291+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd368062160 con 0x7fd368139f50 2020-11-20T13:56:15.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.292+0000 7fd3567fc700 1 -- 172.21.15.117:0/1829157825 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7fd36002b070 con 0x7fd368139f50 2020-11-20T13:56:15.294 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:15.294 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:15.295 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd348051320 msgr2=0x7fd3480537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd348051320 0x7fd3480537e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fd35800d000 tx=0x7fd35800c040).stop 2020-11-20T13:56:15.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368139f50 msgr2=0x7fd36813a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368139f50 0x7fd36813a400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fd36000c000 tx=0x7fd360019000).stop 2020-11-20T13:56:15.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 shutdown_connections 2020-11-20T13:56:15.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd348051320 0x7fd3480537e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.297 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd368139f50 0x7fd36813a400 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.298 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd350004e40 0x7fd368066b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.298 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 --2- 172.21.15.117:0/1829157825 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd36809e7b0 0x7fd36813d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.299 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 >> 172.21.15.117:0/1829157825 conn(0x7fd3680b9470 msgr2=0x7fd3680af470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.299 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 shutdown_connections 2020-11-20T13:56:15.299 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.294+0000 7fd36e2ad700 1 -- 172.21.15.117:0/1829157825 wait complete. 2020-11-20T13:56:15.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:15.308 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:15.420 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.416+0000 7fe32fe86700 1 Processor -- start 2020-11-20T13:56:15.420 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.417+0000 7fe32fe86700 1 -- start start 2020-11-20T13:56:15.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.417+0000 7fe32fe86700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe32809cb40 0x7fe3281437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.417+0000 7fe32fe86700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe328143c90 0x7fe3280b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.417+0000 7fe32fe86700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3280999d0 con 0x7fe32809bd80 2020-11-20T13:56:15.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.417+0000 7fe32fe86700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3280997f0 con 0x7fe32809cb40 2020-11-20T13:56:15.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.417+0000 7fe32fe86700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe328099c70 con 0x7fe328143c90 2020-11-20T13:56:15.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32d421700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe32809cb40 0x7fe3281437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32e423700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe328143c90 0x7fe3280b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32dc22700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7fe32809bd80 0x7fe32809a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:44146/0 (socket says 172.21.15.117:44146) 2020-11-20T13:56:15.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32dc22700 1 -- 172.21.15.117:0/3990299940 learned_addr learned my addr 172.21.15.117:0/3990299940 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:15.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32d421700 1 -- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe328143c90 msgr2=0x7fe3280b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32d421700 1 --2- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe328143c90 0x7fe3280b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32d421700 1 -- 172.21.15.117:0/3990299940 >> v1:172.21.15.85:6789/0 conn(0x7fe32809bd80 legacy=0x7fe32809a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32d421700 1 -- 172.21.15.117:0/3990299940 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328144120 con 0x7fe32809cb40 2020-11-20T13:56:15.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32cc20700 1 -- 172.21.15.117:0/3990299940 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408258573 0 0) 0x7fe3280999d0 con 0x7fe32809bd80 2020-11-20T13:56:15.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32e423700 1 --2- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe328143c90 0x7fe3280b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:15.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32d421700 1 --2- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe32809cb40 0x7fe3281437a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe324002f30 tx=0x7fe324006dc0).ready entity=mon.0 client_cookie=ac514582e612ded0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32cc20700 1 -- 172.21.15.117:0/3990299940 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe324009760 con 0x7fe32809cb40 2020-11-20T13:56:15.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32cc20700 1 -- 172.21.15.117:0/3990299940 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3240098c0 con 0x7fe32809cb40 2020-11-20T13:56:15.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32cc20700 1 -- 172.21.15.117:0/3990299940 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe324009c20 con 0x7fe32809cb40 2020-11-20T13:56:15.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32fe86700 1 -- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe32809cb40 msgr2=0x7fe3281437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.418+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe32809cb40 0x7fe3281437a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe324002f30 tx=0x7fe324006dc0).stop 2020-11-20T13:56:15.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.419+0000 7fe32fe86700 1 -- 172.21.15.117:0/3990299940 shutdown_connections 2020-11-20T13:56:15.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.419+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe328143c90 0x7fe3280b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.419+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3990299940 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe32809cb40 0x7fe3281437a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.430 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.419+0000 7fe32fe86700 1 -- 172.21.15.117:0/3990299940 >> 172.21.15.117:0/3990299940 conn(0x7fe3280b9770 msgr2=0x7fe3280b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.430 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.419+0000 7fe32fe86700 1 -- 172.21.15.117:0/3990299940 shutdown_connections 2020-11-20T13:56:15.430 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.419+0000 7fe32fe86700 1 -- 172.21.15.117:0/3990299940 wait complete. 2020-11-20T13:56:15.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.420+0000 7fe32fe86700 1 Processor -- start 2020-11-20T13:56:15.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.420+0000 7fe32fe86700 1 -- start start 2020-11-20T13:56:15.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.420+0000 7fe32fe86700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe32809bd80 0x7fe3280b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.420+0000 7fe32fe86700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32809cb40 0x7fe3281574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.420+0000 7fe32fe86700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe328143c90 0x7fe328134810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32fe86700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281579b0 con 0x7fe328143c90 2020-11-20T13:56:15.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32fe86700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3280ae6d0 con 0x7fe32809cb40 2020-11-20T13:56:15.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32fe86700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe328157bd0 con 0x7fe32809bd80 2020-11-20T13:56:15.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32809cb40 0x7fe3281574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32809cb40 0x7fe3281574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41084/0 (socket says 172.21.15.117:41084) 2020-11-20T13:56:15.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 -- 172.21.15.117:0/3332262819 learned_addr learned my addr 172.21.15.117:0/3332262819 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:15.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32dc22700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe32809bd80 0x7fe3280b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32e423700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe328143c90 0x7fe328134810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 -- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe32809bd80 msgr2=0x7fe3280b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe32809bd80 0x7fe3280b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 -- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe328143c90 msgr2=0x7fe328134810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe328143c90 0x7fe328134810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 -- 172.21.15.117:0/3332262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328157e70 con 0x7fe32809cb40 2020-11-20T13:56:15.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32d421700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32809cb40 0x7fe3281574c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fe3280b12c0 tx=0x7fe324015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe316ffd700 1 -- 172.21.15.117:0/3332262819 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3240119f0 con 0x7fe32809cb40 2020-11-20T13:56:15.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe316ffd700 1 -- 172.21.15.117:0/3332262819 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe324011b50 con 0x7fe32809cb40 2020-11-20T13:56:15.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe328158110 con 0x7fe32809cb40 2020-11-20T13:56:15.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe316ffd700 1 -- 172.21.15.117:0/3332262819 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe32401d440 con 0x7fe32809cb40 2020-11-20T13:56:15.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.421+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3280b0cf0 con 0x7fe32809cb40 2020-11-20T13:56:15.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.422+0000 7fe316ffd700 1 -- 172.21.15.117:0/3332262819 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe324024030 con 0x7fe32809cb40 2020-11-20T13:56:15.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.422+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3281582a0 con 0x7fe32809cb40 2020-11-20T13:56:15.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.422+0000 7fe316ffd700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe308051210 0x7fe3080536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.423+0000 7fe316ffd700 1 -- 172.21.15.117:0/3332262819 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe324069e20 con 0x7fe32809cb40 2020-11-20T13:56:15.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.423+0000 7fe32dc22700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe308051210 0x7fe3080536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.423+0000 7fe32dc22700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe308051210 0x7fe3080536d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe31800d000 tx=0x7fe318007910).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.426+0000 7fe316ffd700 1 -- 172.21.15.117:0/3332262819 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe324042040 con 0x7fe32809cb40 2020-11-20T13:56:15.577 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.575+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe328057050 con 0x7fe32809cb40 2020-11-20T13:56:15.577 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.576+0000 7fe316ffd700 1 -- 172.21.15.117:0/3332262819 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe324022020 con 0x7fe32809cb40 2020-11-20T13:56:15.577 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:15.579 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe308051210 msgr2=0x7fe3080536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.579 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe308051210 0x7fe3080536d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe31800d000 tx=0x7fe318007910).stop 2020-11-20T13:56:15.579 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32809cb40 msgr2=0x7fe3281574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32809cb40 0x7fe3281574c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fe3280b12c0 tx=0x7fe324015000).stop 2020-11-20T13:56:15.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 shutdown_connections 2020-11-20T13:56:15.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe308051210 0x7fe3080536d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe32809bd80 0x7fe3280b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe328143c90 0x7fe328134810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 --2- 172.21.15.117:0/3332262819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32809cb40 0x7fe3281574c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.577+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 >> 172.21.15.117:0/3332262819 conn(0x7fe3280b9770 msgr2=0x7fe32809f090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.578+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 shutdown_connections 2020-11-20T13:56:15.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.578+0000 7fe32fe86700 1 -- 172.21.15.117:0/3332262819 wait complete. 2020-11-20T13:56:15.589 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:15.698 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.696+0000 7f10a8312700 1 Processor -- start 2020-11-20T13:56:15.698 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.696+0000 7f10a8312700 1 -- start start 2020-11-20T13:56:15.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.697+0000 7f10a8312700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 0x7f10a009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.697+0000 7f10a8312700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00999d0 con 0x7f10a0143eb0 2020-11-20T13:56:15.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.697+0000 7f10a8312700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a0099c70 con 0x7f10a009a110 2020-11-20T13:56:15.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.697+0000 7f10a8312700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a009c160 con 0x7f10a009bd80 2020-11-20T13:56:15.702 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a58ad700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f10a009a110 0x7f10a01437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60252/0 (socket says 172.21.15.117:60252) 2020-11-20T13:56:15.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a58ad700 1 -- 172.21.15.117:0/3060604533 learned_addr learned my addr 172.21.15.117:0/3060604533 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:15.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a60ae700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 0x7f10a009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440329529 0 0) 0x7f10a0099c70 con 0x7f10a009a110 2020-11-20T13:56:15.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1088003230 con 0x7f10a009a110 2020-11-20T13:56:15.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 609813262 0 0) 0x7f1088003230 con 0x7f10a009a110 2020-11-20T13:56:15.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 msgr2=0x7f10a009cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 0x7f10a009cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 >> v1:172.21.15.85:6789/0 conn(0x7f10a0143eb0 legacy=0x7f10a00b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0098cb0 con 0x7f10a009a110 2020-11-20T13:56:15.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f109c0016d0 con 0x7f10a009a110 2020-11-20T13:56:15.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 >> v1:172.21.15.117:6789/0 conn(0x7f10a009a110 legacy=0x7f10a01437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1088003750 0x7f1088003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 0x7f10880044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10880049c0 0x7f1088004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a009c160 con 0x7f1088003750 2020-11-20T13:56:15.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10880041f0 con 0x7f10880049c0 2020-11-20T13:56:15.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1088004330 con 0x7f10a009bd80 2020-11-20T13:56:15.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a68af700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10880049c0 0x7f1088004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a58ad700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 0x7f10880044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a60ae700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1088003750 0x7f1088003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a68af700 1 -- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 msgr2=0x7f10880044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a68af700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a009bd80 0x7f10880044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a68af700 1 -- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1088003750 msgr2=0x7f1088003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a68af700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1088003750 0x7f1088003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.698+0000 7f10a68af700 1 -- 172.21.15.117:0/3060604533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088003230 con 0x7f10880049c0 2020-11-20T13:56:15.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a68af700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10880049c0 0x7f1088004e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f10880054a0 tx=0x7f1098007e50).ready entity=mon.1 client_cookie=b06891ef00ea3e3a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109800ca10 con 0x7f10880049c0 2020-11-20T13:56:15.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109800cb70 con 0x7f10880049c0 2020-11-20T13:56:15.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a50ac700 1 -- 172.21.15.117:0/3060604533 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1098012400 con 0x7f10880049c0 2020-11-20T13:56:15.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a8312700 1 -- 172.21.15.117:0/3060604533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10880049c0 msgr2=0x7f1088004e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a8312700 1 --2- 172.21.15.117:0/3060604533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10880049c0 0x7f1088004e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f10880054a0 tx=0x7f1098007e50).stop 2020-11-20T13:56:15.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a68af700 1 -- 172.21.15.117:0/3060604533 reap_dead start 2020-11-20T13:56:15.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a8312700 1 -- 172.21.15.117:0/3060604533 shutdown_connections 2020-11-20T13:56:15.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a8312700 1 -- 172.21.15.117:0/3060604533 >> 172.21.15.117:0/3060604533 conn(0x7f10a00b9770 msgr2=0x7f10a00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a8312700 1 -- 172.21.15.117:0/3060604533 shutdown_connections 2020-11-20T13:56:15.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.699+0000 7f10a8312700 1 -- 172.21.15.117:0/3060604533 wait complete. 2020-11-20T13:56:15.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.700+0000 7f10a8312700 1 Processor -- start 2020-11-20T13:56:15.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.700+0000 7f10a8312700 1 -- start start 2020-11-20T13:56:15.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10a00645f0 0x7f10a015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a0066ed0 0x7f10a00673f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a01429e0 0x7f10a0142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00678e0 con 0x7f10a00645f0 2020-11-20T13:56:15.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01596e0 con 0x7f10a01429e0 2020-11-20T13:56:15.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0159980 con 0x7f10a0066ed0 2020-11-20T13:56:15.720 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a01429e0 0x7f10a0142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a01429e0 0x7f10a0142e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41106/0 (socket says 172.21.15.117:41106) 2020-11-20T13:56:15.721 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 -- 172.21.15.117:0/4203453701 learned_addr learned my addr 172.21.15.117:0/4203453701 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:15.722 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a60ae700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10a00645f0 0x7f10a015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.722 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a58ad700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a0066ed0 0x7f10a00673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.722 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 -- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a0066ed0 msgr2=0x7f10a00673f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.722 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a0066ed0 0x7f10a00673f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 -- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10a00645f0 msgr2=0x7f10a015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10a00645f0 0x7f10a015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 -- 172.21.15.117:0/4203453701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0159c20 con 0x7f10a01429e0 2020-11-20T13:56:15.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a68af700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a01429e0 0x7f10a0142e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f10a009b8e0 tx=0x7f1098015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.723 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f108f7fe700 1 -- 172.21.15.117:0/4203453701 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1098022070 con 0x7f10a01429e0 2020-11-20T13:56:15.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f108f7fe700 1 -- 172.21.15.117:0/4203453701 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109801d790 con 0x7f10a01429e0 2020-11-20T13:56:15.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a00649d0 con 0x7f10a01429e0 2020-11-20T13:56:15.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f108f7fe700 1 -- 172.21.15.117:0/4203453701 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f109801daf0 con 0x7f10a01429e0 2020-11-20T13:56:15.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.701+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a00ae910 con 0x7f10a01429e0 2020-11-20T13:56:15.724 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.703+0000 7f108f7fe700 1 -- 172.21.15.117:0/4203453701 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1098010070 con 0x7f10a01429e0 2020-11-20T13:56:15.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.703+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0159db0 con 0x7f10a01429e0 2020-11-20T13:56:15.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.703+0000 7f108f7fe700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f10800513e0 0x7f10800538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.703+0000 7f108f7fe700 1 -- 172.21.15.117:0/4203453701 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f109806a610 con 0x7f10a01429e0 2020-11-20T13:56:15.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.703+0000 7f10a60ae700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f10800513e0 0x7f10800538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.725 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.703+0000 7f10a60ae700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f10800513e0 0x7f10800538a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1090008000 tx=0x7f1090008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.726 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.706+0000 7f108f7fe700 1 -- 172.21.15.117:0/4203453701 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f10980428b0 con 0x7f10a01429e0 2020-11-20T13:56:15.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.856+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f10a0057050 con 0x7f10a01429e0 2020-11-20T13:56:15.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.857+0000 7f108f7fe700 1 -- 172.21.15.117:0/4203453701 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1098017020 con 0x7f10a01429e0 2020-11-20T13:56:15.859 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:15.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f10800513e0 msgr2=0x7f10800538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f10800513e0 0x7f10800538a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1090008000 tx=0x7f1090008040).stop 2020-11-20T13:56:15.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a01429e0 msgr2=0x7f10a0142e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a01429e0 0x7f10a0142e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f10a009b8e0 tx=0x7f1098015000).stop 2020-11-20T13:56:15.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 shutdown_connections 2020-11-20T13:56:15.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f10800513e0 0x7f10800538a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a01429e0 0x7f10a0142e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f10a00645f0 0x7f10a015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 --2- 172.21.15.117:0/4203453701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f10a0066ed0 0x7f10a00673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 >> 172.21.15.117:0/4203453701 conn(0x7f10a00b9770 msgr2=0x7f10a00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 shutdown_connections 2020-11-20T13:56:15.862 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.858+0000 7f10a8312700 1 -- 172.21.15.117:0/4203453701 wait complete. 2020-11-20T13:56:15.870 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:15.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.978+0000 7f9f99de2700 1 Processor -- start 2020-11-20T13:56:15.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.978+0000 7f9f99de2700 1 -- start start 2020-11-20T13:56:15.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f99de2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9f9405f300 0x7f9f94062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f99de2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f9409c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f99de2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9409f100 con 0x7f9f9409bf70 2020-11-20T13:56:15.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f99de2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9409eef0 con 0x7f9f9405f300 2020-11-20T13:56:15.984 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f99de2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9409f3a0 con 0x7f9f94147030 2020-11-20T13:56:15.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f98de0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9f9405f300 0x7f9f94062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f93fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9f9409bf70 0x7f9f94146920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60270/0 (socket says 172.21.15.117:60270) 2020-11-20T13:56:15.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f93fff700 1 -- 172.21.15.117:0/3205838120 learned_addr learned my addr 172.21.15.117:0/3205838120 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:15.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.979+0000 7f9f995e1700 1 --2- 172.21.15.117:0/3205838120 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f9409c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f98de0700 1 -- 172.21.15.117:0/3205838120 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 msgr2=0x7f9f9409c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:15.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f98de0700 1 --2- 172.21.15.117:0/3205838120 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f9409c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f98de0700 1 -- 172.21.15.117:0/3205838120 >> v1:172.21.15.117:6789/0 conn(0x7f9f9409bf70 legacy=0x7f9f94146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f937fe700 1 -- 172.21.15.117:0/3205838120 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594861856 0 0) 0x7f9f9409f100 con 0x7f9f9409bf70 2020-11-20T13:56:15.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f98de0700 1 -- 172.21.15.117:0/3205838120 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9405f6e0 con 0x7f9f9405f300 2020-11-20T13:56:15.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f98de0700 1 --2- 172.21.15.117:0/3205838120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f94062b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f9f84002f30 tx=0x7f9f84006dc0).ready entity=mon.1 client_cookie=4d457745e21ea218 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f937fe700 1 -- 172.21.15.117:0/3205838120 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f8400f9c0 con 0x7f9f9405f300 2020-11-20T13:56:15.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f937fe700 1 -- 172.21.15.117:0/3205838120 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8400fb20 con 0x7f9f9405f300 2020-11-20T13:56:15.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f937fe700 1 -- 172.21.15.117:0/3205838120 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f8400fe80 con 0x7f9f9405f300 2020-11-20T13:56:15.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f99de2700 1 -- 172.21.15.117:0/3205838120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 msgr2=0x7f9f94062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f99de2700 1 --2- 172.21.15.117:0/3205838120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f94062b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f9f84002f30 tx=0x7f9f84006dc0).stop 2020-11-20T13:56:15.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f99de2700 1 -- 172.21.15.117:0/3205838120 shutdown_connections 2020-11-20T13:56:15.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f99de2700 1 --2- 172.21.15.117:0/3205838120 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f9409c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f99de2700 1 --2- 172.21.15.117:0/3205838120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f94062b50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f99de2700 1 -- 172.21.15.117:0/3205838120 >> 172.21.15.117:0/3205838120 conn(0x7f9f940b9490 msgr2=0x7f9f940b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:15.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.980+0000 7f9f99de2700 1 -- 172.21.15.117:0/3205838120 shutdown_connections 2020-11-20T13:56:15.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.981+0000 7f9f99de2700 1 -- 172.21.15.117:0/3205838120 wait complete. 2020-11-20T13:56:15.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.981+0000 7f9f99de2700 1 Processor -- start 2020-11-20T13:56:15.990 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.981+0000 7f9f99de2700 1 -- start start 2020-11-20T13:56:15.992 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f99de2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f940b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.992 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f99de2700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9f9409bf70 0x7f9f940b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.992 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f99de2700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f940655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.993 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f99de2700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b32e0 con 0x7f9f9409bf70 2020-11-20T13:56:15.993 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f99de2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b2170 con 0x7f9f9405f300 2020-11-20T13:56:15.993 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f99de2700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940b2410 con 0x7f9f94147030 2020-11-20T13:56:15.993 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f98de0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f940b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f98de0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f940b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41114/0 (socket says 172.21.15.117:41114) 2020-11-20T13:56:15.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f98de0700 1 -- 172.21.15.117:0/2745519932 learned_addr learned my addr 172.21.15.117:0/2745519932 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:15.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f93fff700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9f9409bf70 0x7f9f940b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.982+0000 7f9f995e1700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f940655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.994 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f98de0700 1 -- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 msgr2=0x7f9f940655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f98de0700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f940655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f98de0700 1 -- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9f9409bf70 msgr2=0x7f9f940b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f98de0700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9f9409bf70 0x7f9f940b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f98de0700 1 -- 172.21.15.117:0/2745519932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f940b26b0 con 0x7f9f9405f300 2020-11-20T13:56:15.995 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f995e1700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f940655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f98de0700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f940b4d60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f9f84002f30 tx=0x7f9f84019000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f91ffb700 1 -- 172.21.15.117:0/2745519932 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f84015c30 con 0x7f9f9405f300 2020-11-20T13:56:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f91ffb700 1 -- 172.21.15.117:0/2745519932 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f84015d90 con 0x7f9f9405f300 2020-11-20T13:56:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f940b2950 con 0x7f9f9405f300 2020-11-20T13:56:15.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f91ffb700 1 -- 172.21.15.117:0/2745519932 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f84010660 con 0x7f9f9405f300 2020-11-20T13:56:15.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.983+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f940ae600 con 0x7f9f9405f300 2020-11-20T13:56:15.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.984+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f940b2ae0 con 0x7f9f9405f300 2020-11-20T13:56:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.984+0000 7f9f91ffb700 1 -- 172.21.15.117:0/2745519932 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9f84028030 con 0x7f9f9405f300 2020-11-20T13:56:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.984+0000 7f9f91ffb700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f740513c0 0x7f9f74053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.984+0000 7f9f91ffb700 1 -- 172.21.15.117:0/2745519932 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f9f8406e2b0 con 0x7f9f9405f300 2020-11-20T13:56:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.984+0000 7f9f93fff700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f740513c0 0x7f9f74053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:15.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.985+0000 7f9f93fff700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f740513c0 0x7f9f74053880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f9f8800d000 tx=0x7f9f88001160).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:15.988+0000 7f9f91ffb700 1 -- 172.21.15.117:0/2745519932 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9f840464d0 con 0x7f9f9405f300 2020-11-20T13:56:16.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.136+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f94057050 con 0x7f9f9405f300 2020-11-20T13:56:16.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.136+0000 7f9f91ffb700 1 -- 172.21.15.117:0/2745519932 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9f84026020 con 0x7f9f9405f300 2020-11-20T13:56:16.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.137+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f740513c0 msgr2=0x7f9f74053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.137+0000 7f9f99de2700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f740513c0 0x7f9f74053880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f9f8800d000 tx=0x7f9f88001160).stop 2020-11-20T13:56:16.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.137+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 msgr2=0x7f9f940b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.137+0000 7f9f99de2700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f940b4d60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f9f84002f30 tx=0x7f9f84019000).stop 2020-11-20T13:56:16.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 shutdown_connections 2020-11-20T13:56:16.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f740513c0 0x7f9f74053880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9f94147030 0x7f9f940655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f9405f300 0x7f9f940b4d60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 --2- 172.21.15.117:0/2745519932 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9f9409bf70 0x7f9f940b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 >> 172.21.15.117:0/2745519932 conn(0x7f9f940b9490 msgr2=0x7f9f9409c580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:16.142 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 shutdown_connections 2020-11-20T13:56:16.142 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.138+0000 7f9f99de2700 1 -- 172.21.15.117:0/2745519932 wait complete. 2020-11-20T13:56:16.142 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:16.149 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:16.259 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 Processor -- start 2020-11-20T13:56:16.259 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 -- start start 2020-11-20T13:56:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009bd80 0x7f98e009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f98e009a110 0x7f98e01437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.262 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98e0143c90 0x7f98e00b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e00999d0 con 0x7f98e009a110 2020-11-20T13:56:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e0099c70 con 0x7f98e0143c90 2020-11-20T13:56:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.258+0000 7f98e66d0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0144120 con 0x7f98e009bd80 2020-11-20T13:56:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98e0143c90 0x7f98e00b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.263 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98e0143c90 0x7f98e00b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41126/0 (socket says 172.21.15.117:41126) 2020-11-20T13:56:16.264 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 -- 172.21.15.117:0/2407060487 learned_addr learned my addr 172.21.15.117:0/2407060487 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:16.264 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98df7fe700 1 --2- 172.21.15.117:0/2407060487 >> v2:172.21.15.85:3300/0 conn(0x7f98e009a110 0x7f98e01437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.264 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98dffff700 1 --2- 172.21.15.117:0/2407060487 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009bd80 0x7f98e009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.264 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 -- 172.21.15.117:0/2407060487 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009bd80 msgr2=0x7f98e009cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:16.265 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 --2- 172.21.15.117:0/2407060487 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009bd80 0x7f98e009cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.265 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 -- 172.21.15.117:0/2407060487 >> v2:172.21.15.85:3300/0 conn(0x7f98e009a110 msgr2=0x7f98e01437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:16.266 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 --2- 172.21.15.117:0/2407060487 >> v2:172.21.15.85:3300/0 conn(0x7f98e009a110 0x7f98e01437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.266 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 -- 172.21.15.117:0/2407060487 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e009c160 con 0x7f98e0143c90 2020-11-20T13:56:16.266 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98dffff700 1 --2- 172.21.15.117:0/2407060487 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009bd80 0x7f98e009cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:16.267 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e4c6d700 1 --2- 172.21.15.117:0/2407060487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0143c90 0x7f98e00b4110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f98d80032b0 tx=0x7f98d8009270).ready entity=mon.1 client_cookie=e35f495dfc1a0a89 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.267 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98deffd700 1 -- 172.21.15.117:0/2407060487 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d8009e60 con 0x7f98e0143c90 2020-11-20T13:56:16.267 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98deffd700 1 -- 172.21.15.117:0/2407060487 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d800bc00 con 0x7f98e0143c90 2020-11-20T13:56:16.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98deffd700 1 -- 172.21.15.117:0/2407060487 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d800bd70 con 0x7f98e0143c90 2020-11-20T13:56:16.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e66d0700 1 -- 172.21.15.117:0/2407060487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0143c90 msgr2=0x7f98e00b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.259+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2407060487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0143c90 0x7f98e00b4110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f98d80032b0 tx=0x7f98d8009270).stop 2020-11-20T13:56:16.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.260+0000 7f98e66d0700 1 -- 172.21.15.117:0/2407060487 shutdown_connections 2020-11-20T13:56:16.269 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.260+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2407060487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e0143c90 0x7f98e00b4110 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.269 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.260+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2407060487 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009bd80 0x7f98e009cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.269 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.260+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2407060487 >> v2:172.21.15.85:3300/0 conn(0x7f98e009a110 0x7f98e01437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.269 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.260+0000 7f98e66d0700 1 -- 172.21.15.117:0/2407060487 >> 172.21.15.117:0/2407060487 conn(0x7f98e00b9770 msgr2=0x7f98e00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:16.269 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.260+0000 7f98e66d0700 1 -- 172.21.15.117:0/2407060487 shutdown_connections 2020-11-20T13:56:16.270 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.260+0000 7f98e66d0700 1 -- 172.21.15.117:0/2407060487 wait complete. 2020-11-20T13:56:16.270 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.261+0000 7f98e66d0700 1 Processor -- start 2020-11-20T13:56:16.270 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.261+0000 7f98e66d0700 1 -- start start 2020-11-20T13:56:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.261+0000 7f98e66d0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009a110 0x7f98e0066ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.261+0000 7f98e66d0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e009bd80 0x7f98e0141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.274 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.261+0000 7f98e66d0700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98e0143c90 0x7f98e0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.261+0000 7f98e66d0700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0066d50 con 0x7f98e0143c90 2020-11-20T13:56:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98e66d0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0152e40 con 0x7f98e009bd80 2020-11-20T13:56:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98e66d0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0153080 con 0x7f98e009a110 2020-11-20T13:56:16.276 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e009bd80 0x7f98e0141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e009bd80 0x7f98e0141970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41130/0 (socket says 172.21.15.117:41130) 2020-11-20T13:56:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 -- 172.21.15.117:0/2671671691 learned_addr learned my addr 172.21.15.117:0/2671671691 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98dffff700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009a110 0x7f98e0066ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98e4c6d700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98e0143c90 0x7f98e0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.277 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 -- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009a110 msgr2=0x7f98e0066ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009a110 0x7f98e0066ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 -- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98e0143c90 msgr2=0x7f98e0156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98e0143c90 0x7f98e0156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 -- 172.21.15.117:0/2671671691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0153320 con 0x7f98e009bd80 2020-11-20T13:56:16.278 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98df7fe700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e009bd80 0x7f98e0141970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f98d400c000 tx=0x7f98d4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98dd7fa700 1 -- 172.21.15.117:0/2671671691 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d400ca10 con 0x7f98e009bd80 2020-11-20T13:56:16.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98dd7fa700 1 -- 172.21.15.117:0/2671671691 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400cb70 con 0x7f98e009bd80 2020-11-20T13:56:16.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e01537c0 con 0x7f98e009bd80 2020-11-20T13:56:16.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98dd7fa700 1 -- 172.21.15.117:0/2671671691 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98d4011450 con 0x7f98e009bd80 2020-11-20T13:56:16.279 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.262+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e0153d60 con 0x7f98e009bd80 2020-11-20T13:56:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.263+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e0157270 con 0x7f98e009bd80 2020-11-20T13:56:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.263+0000 7f98dd7fa700 1 -- 172.21.15.117:0/2671671691 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f98d40115b0 con 0x7f98e009bd80 2020-11-20T13:56:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.263+0000 7f98dd7fa700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98c00513c0 0x7f98c0053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.264+0000 7f98dd7fa700 1 -- 172.21.15.117:0/2671671691 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f98d406aee0 con 0x7f98e009bd80 2020-11-20T13:56:16.281 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.264+0000 7f98dffff700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98c00513c0 0x7f98c0053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.281 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.264+0000 7f98dffff700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98c00513c0 0x7f98c0053880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f98d000c000 tx=0x7f98d0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.281 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.267+0000 7f98dd7fa700 1 -- 172.21.15.117:0/2671671691 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f98d40430a0 con 0x7f98e009bd80 2020-11-20T13:56:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.420+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98e0153950 con 0x7f98e009bd80 2020-11-20T13:56:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.420+0000 7f98dd7fa700 1 -- 172.21.15.117:0/2671671691 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f98e0153950 con 0x7f98e009bd80 2020-11-20T13:56:16.422 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:16.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.421+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98c00513c0 msgr2=0x7f98c0053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.421+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98c00513c0 0x7f98c0053880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f98d000c000 tx=0x7f98d0008040).stop 2020-11-20T13:56:16.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.421+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e009bd80 msgr2=0x7f98e0141970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.421+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e009bd80 0x7f98e0141970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f98d400c000 tx=0x7f98d4006dc0).stop 2020-11-20T13:56:16.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 shutdown_connections 2020-11-20T13:56:16.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f98c00513c0 0x7f98c0053880 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f98e009a110 0x7f98e0066ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f98e0143c90 0x7f98e0156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 --2- 172.21.15.117:0/2671671691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e009bd80 0x7f98e0141970 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 >> 172.21.15.117:0/2671671691 conn(0x7f98e00b9770 msgr2=0x7f98e01423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:16.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 shutdown_connections 2020-11-20T13:56:16.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.422+0000 7f98e66d0700 1 -- 172.21.15.117:0/2671671691 wait complete. 2020-11-20T13:56:16.433 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:16.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.540+0000 7f3709b03700 1 Processor -- start 2020-11-20T13:56:16.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.540+0000 7f3709b03700 1 -- start start 2020-11-20T13:56:16.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f3709b03700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f3704098e30 0x7f370409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f3709b03700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37040b5510 con 0x7f370409b700 2020-11-20T13:56:16.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f3709b03700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37040b57b0 con 0x7f37040ae6f0 2020-11-20T13:56:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f3709b03700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37040b5a50 con 0x7f3704098e30 2020-11-20T13:56:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f3703fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f37040ae6f0 0x7f37040aeb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60296/0 (socket says 172.21.15.117:60296) 2020-11-20T13:56:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f3703fff700 1 -- 172.21.15.117:0/419975015 learned_addr learned my addr 172.21.15.117:0/419975015 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978053339 0 0) 0x7f37040b57b0 con 0x7f37040ae6f0 2020-11-20T13:56:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f0003230 con 0x7f37040ae6f0 2020-11-20T13:56:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37037fe700 1 --2- 172.21.15.117:0/419975015 >> v2:172.21.15.85:3300/0 conn(0x7f3704098e30 0x7f370409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2897192929 0 0) 0x7f36f0003230 con 0x7f37040ae6f0 2020-11-20T13:56:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 >> v2:172.21.15.85:3300/0 conn(0x7f3704098e30 msgr2=0x7f370409b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 --2- 172.21.15.117:0/419975015 >> v2:172.21.15.85:3300/0 conn(0x7f3704098e30 0x7f370409b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 >> v1:172.21.15.85:6789/0 conn(0x7f370409b700 legacy=0x7f37041438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704099210 con 0x7f37040ae6f0 2020-11-20T13:56:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.541+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f36fc001a90 con 0x7f37040ae6f0 2020-11-20T13:56:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 >> v1:172.21.15.117:6789/0 conn(0x7f37040ae6f0 legacy=0x7f37040aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f0003770 0x7f36f0003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36f00044d0 0x7f36f0004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.549 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36f0004e40 0x7f36f000b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040b5a50 con 0x7f36f0004e40 2020-11-20T13:56:16.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f000bc10 con 0x7f36f00044d0 2020-11-20T13:56:16.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f000bd50 con 0x7f36f0003770 2020-11-20T13:56:16.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3702ffd700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36f00044d0 0x7f36f0004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37037fe700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f0003770 0x7f36f0003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3703fff700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36f0004e40 0x7f36f000b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3702ffd700 1 -- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f0003770 msgr2=0x7f36f0003b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:16.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3702ffd700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f0003770 0x7f36f0003b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3702ffd700 1 -- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36f0004e40 msgr2=0x7f36f000b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:16.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3702ffd700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36f0004e40 0x7f36f000b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3702ffd700 1 -- 172.21.15.117:0/419975015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f0003230 con 0x7f36f00044d0 2020-11-20T13:56:16.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3703fff700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f36f0004e40 0x7f36f000b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:16.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3703fff700 1 -- 172.21.15.117:0/419975015 reap_dead start 2020-11-20T13:56:16.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3702ffd700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36f00044d0 0x7f36f0004950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f3704060b70 tx=0x7f36f8007b60).ready entity=mon.1 client_cookie=c3c5366601db44c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36f800ca60 con 0x7f36f00044d0 2020-11-20T13:56:16.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f800cbc0 con 0x7f36f00044d0 2020-11-20T13:56:16.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f37027fc700 1 -- 172.21.15.117:0/419975015 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36f8011470 con 0x7f36f00044d0 2020-11-20T13:56:16.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3709b03700 1 -- 172.21.15.117:0/419975015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36f00044d0 msgr2=0x7f36f0004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.542+0000 7f3709b03700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36f00044d0 0x7f36f0004950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f3704060b70 tx=0x7f36f8007b60).stop 2020-11-20T13:56:16.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.543+0000 7f3709b03700 1 -- 172.21.15.117:0/419975015 shutdown_connections 2020-11-20T13:56:16.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.543+0000 7f3709b03700 1 --2- 172.21.15.117:0/419975015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36f00044d0 0x7f36f0004950 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.543+0000 7f3709b03700 1 -- 172.21.15.117:0/419975015 >> 172.21.15.117:0/419975015 conn(0x7f370405b9f0 msgr2=0x7f370405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:16.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.543+0000 7f3709b03700 1 -- 172.21.15.117:0/419975015 shutdown_connections 2020-11-20T13:56:16.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.543+0000 7f3709b03700 1 -- 172.21.15.117:0/419975015 wait complete. 2020-11-20T13:56:16.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.544+0000 7f3709b03700 1 Processor -- start 2020-11-20T13:56:16.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.544+0000 7f3709b03700 1 -- start start 2020-11-20T13:56:16.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.544+0000 7f3709b03700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f00044d0 0x7f3704145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3709b03700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37041554d0 0x7f3704146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3709b03700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f370409e970 0x7f370409edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3709b03700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040b3180 con 0x7f370409e970 2020-11-20T13:56:16.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3709b03700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3704145b60 con 0x7f37041554d0 2020-11-20T13:56:16.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3709b03700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3704145e00 con 0x7f36f00044d0 2020-11-20T13:56:16.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37041554d0 0x7f3704146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37041554d0 0x7f3704146b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41148/0 (socket says 172.21.15.117:41148) 2020-11-20T13:56:16.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 -- 172.21.15.117:0/4272004211 learned_addr learned my addr 172.21.15.117:0/4272004211 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:16.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f37037fe700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f00044d0 0x7f3704145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3703fff700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f370409e970 0x7f370409edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 -- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f00044d0 msgr2=0x7f3704145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f00044d0 0x7f3704145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 -- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f370409e970 msgr2=0x7f370409edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f370409e970 0x7f370409edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 -- 172.21.15.117:0/4272004211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37041460a0 con 0x7f37041554d0 2020-11-20T13:56:16.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3702ffd700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37041554d0 0x7f3704146b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f3704145a80 tx=0x7f36f8018000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3700ff9700 1 -- 172.21.15.117:0/4272004211 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36f8007890 con 0x7f37041554d0 2020-11-20T13:56:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3700ff9700 1 -- 172.21.15.117:0/4272004211 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8020070 con 0x7f37041554d0 2020-11-20T13:56:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3704146540 con 0x7f37041554d0 2020-11-20T13:56:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3700ff9700 1 -- 172.21.15.117:0/4272004211 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36f8007410 con 0x7f37041554d0 2020-11-20T13:56:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.545+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37041470c0 con 0x7f37041554d0 2020-11-20T13:56:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.546+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3704147620 con 0x7f37041554d0 2020-11-20T13:56:16.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.546+0000 7f3700ff9700 1 -- 172.21.15.117:0/4272004211 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f36f80064e0 con 0x7f37041554d0 2020-11-20T13:56:16.568 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.546+0000 7f3700ff9700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36e4051350 0x7f36e4053810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.568 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.547+0000 7f3700ff9700 1 -- 172.21.15.117:0/4272004211 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f36f806c7f0 con 0x7f37041554d0 2020-11-20T13:56:16.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.547+0000 7f37037fe700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36e4051350 0x7f36e4053810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.547+0000 7f37037fe700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36e4051350 0x7f36e4053810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f36f4008000 tx=0x7f36f4008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.550+0000 7f3700ff9700 1 -- 172.21.15.117:0/4272004211 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f36f8044a60 con 0x7f37041554d0 2020-11-20T13:56:16.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.699+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3704146230 con 0x7f37041554d0 2020-11-20T13:56:16.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.699+0000 7f3700ff9700 1 -- 172.21.15.117:0/4272004211 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f36f8048080 con 0x7f37041554d0 2020-11-20T13:56:16.701 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:16.702 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:16.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36e4051350 msgr2=0x7f36e4053810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36e4051350 0x7f36e4053810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f36f4008000 tx=0x7f36f4008040).stop 2020-11-20T13:56:16.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37041554d0 msgr2=0x7f3704146b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.703 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37041554d0 0x7f3704146b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f3704145a80 tx=0x7f36f8018000).stop 2020-11-20T13:56:16.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 shutdown_connections 2020-11-20T13:56:16.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36e4051350 0x7f36e4053810 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f36f00044d0 0x7f3704145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f370409e970 0x7f370409edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.704 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 --2- 172.21.15.117:0/4272004211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37041554d0 0x7f3704146b50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 >> 172.21.15.117:0/4272004211 conn(0x7f370405b9f0 msgr2=0x7f37040aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:16.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 shutdown_connections 2020-11-20T13:56:16.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.701+0000 7f3709b03700 1 -- 172.21.15.117:0/4272004211 wait complete. 2020-11-20T13:56:16.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:16.713 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:16.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.822+0000 7f3a1957f700 1 Processor -- start 2020-11-20T13:56:16.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.822+0000 7f3a1957f700 1 -- start start 2020-11-20T13:56:16.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a1957f700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f3a1409bd80 0x7f3a1409cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a1957f700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a1409a110 0x7f3a141437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a1957f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a140999d0 con 0x7f3a14143c90 2020-11-20T13:56:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a1957f700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a14099c70 con 0x7f3a1409bd80 2020-11-20T13:56:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a1957f700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1409c160 con 0x7f3a1409a110 2020-11-20T13:56:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a137fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3a14143c90 0x7f3a140b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60314/0 (socket says 172.21.15.117:60314) 2020-11-20T13:56:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a137fe700 1 -- 172.21.15.117:0/1539726649 learned_addr learned my addr 172.21.15.117:0/1539726649 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:16.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a127fc700 1 --2- 172.21.15.117:0/1539726649 >> v2:172.21.15.117:3300/0 conn(0x7f3a1409a110 0x7f3a141437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.823+0000 7f3a12ffd700 1 --2- 172.21.15.117:0/1539726649 >> v2:172.21.15.85:3300/0 conn(0x7f3a1409bd80 0x7f3a1409cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a127fc700 1 -- 172.21.15.117:0/1539726649 >> v2:172.21.15.85:3300/0 conn(0x7f3a1409bd80 msgr2=0x7f3a1409cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:16.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a127fc700 1 --2- 172.21.15.117:0/1539726649 >> v2:172.21.15.85:3300/0 conn(0x7f3a1409bd80 0x7f3a1409cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a127fc700 1 -- 172.21.15.117:0/1539726649 >> v1:172.21.15.117:6789/0 conn(0x7f3a14143c90 legacy=0x7f3a140b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a127fc700 1 -- 172.21.15.117:0/1539726649 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a141440e0 con 0x7f3a1409a110 2020-11-20T13:56:16.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a11ffb700 1 -- 172.21.15.117:0/1539726649 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175201700 0 0) 0x7f3a140999d0 con 0x7f3a14143c90 2020-11-20T13:56:16.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a12ffd700 1 --2- 172.21.15.117:0/1539726649 >> v2:172.21.15.85:3300/0 conn(0x7f3a1409bd80 0x7f3a1409cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:16.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a127fc700 1 --2- 172.21.15.117:0/1539726649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a1409a110 0x7f3a141437a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f3a04002f30 tx=0x7f3a04006dc0).ready entity=mon.1 client_cookie=b7dac43697a76508 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a11ffb700 1 -- 172.21.15.117:0/1539726649 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a040097f0 con 0x7f3a1409a110 2020-11-20T13:56:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a11ffb700 1 -- 172.21.15.117:0/1539726649 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a04009950 con 0x7f3a1409a110 2020-11-20T13:56:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a11ffb700 1 -- 172.21.15.117:0/1539726649 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a04009cb0 con 0x7f3a1409a110 2020-11-20T13:56:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 -- 172.21.15.117:0/1539726649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a1409a110 msgr2=0x7f3a141437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 --2- 172.21.15.117:0/1539726649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a1409a110 0x7f3a141437a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f3a04002f30 tx=0x7f3a04006dc0).stop 2020-11-20T13:56:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 -- 172.21.15.117:0/1539726649 shutdown_connections 2020-11-20T13:56:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 --2- 172.21.15.117:0/1539726649 >> v2:172.21.15.85:3300/0 conn(0x7f3a1409bd80 0x7f3a1409cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 --2- 172.21.15.117:0/1539726649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a1409a110 0x7f3a141437a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 -- 172.21.15.117:0/1539726649 >> 172.21.15.117:0/1539726649 conn(0x7f3a140b9770 msgr2=0x7f3a140b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 -- 172.21.15.117:0/1539726649 shutdown_connections 2020-11-20T13:56:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.824+0000 7f3a1957f700 1 -- 172.21.15.117:0/1539726649 wait complete. 2020-11-20T13:56:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.825+0000 7f3a1957f700 1 Processor -- start 2020-11-20T13:56:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.825+0000 7f3a1957f700 1 -- start start 2020-11-20T13:56:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a1957f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3a1409a110 0x7f3a14140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a1957f700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3a1409bd80 0x7f3a14156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a1957f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a14143c90 0x7f3a14148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a1957f700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14156df0 con 0x7f3a1409bd80 2020-11-20T13:56:16.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a1957f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14157090 con 0x7f3a14143c90 2020-11-20T13:56:16.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a1957f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14159a30 con 0x7f3a1409a110 2020-11-20T13:56:16.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a14143c90 0x7f3a14148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a14143c90 0x7f3a14148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41164/0 (socket says 172.21.15.117:41164) 2020-11-20T13:56:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 -- 172.21.15.117:0/3929797100 learned_addr learned my addr 172.21.15.117:0/3929797100 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a12ffd700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3a1409a110 0x7f3a14140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a127fc700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3a1409bd80 0x7f3a14156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 -- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3a1409a110 msgr2=0x7f3a14140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3a1409a110 0x7f3a14140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 -- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3a1409bd80 msgr2=0x7f3a14156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3a1409bd80 0x7f3a14156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.826+0000 7f3a137fe700 1 -- 172.21.15.117:0/3929797100 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14156960 con 0x7f3a14143c90 2020-11-20T13:56:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.827+0000 7f3a137fe700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a14143c90 0x7f3a14148740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f3a0c000f00 tx=0x7f3a0c007cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.827+0000 7f3a03fff700 1 -- 172.21.15.117:0/3929797100 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0c005960 con 0x7f3a14143c90 2020-11-20T13:56:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.827+0000 7f3a03fff700 1 -- 172.21.15.117:0/3929797100 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c005ac0 con 0x7f3a14143c90 2020-11-20T13:56:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.827+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a14156bb0 con 0x7f3a14143c90 2020-11-20T13:56:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.827+0000 7f3a03fff700 1 -- 172.21.15.117:0/3929797100 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a0c005e20 con 0x7f3a14143c90 2020-11-20T13:56:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.827+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a14148df0 con 0x7f3a14143c90 2020-11-20T13:56:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.828+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a14157290 con 0x7f3a14143c90 2020-11-20T13:56:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.828+0000 7f3a03fff700 1 -- 172.21.15.117:0/3929797100 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3a0c010030 con 0x7f3a14143c90 2020-11-20T13:56:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.828+0000 7f3a03fff700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39f00513c0 0x7f39f0053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.828+0000 7f3a03fff700 1 -- 172.21.15.117:0/3929797100 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f3a0c068de0 con 0x7f3a14143c90 2020-11-20T13:56:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.828+0000 7f3a12ffd700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39f00513c0 0x7f39f0053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.829+0000 7f3a12ffd700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39f00513c0 0x7f39f0053880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f3a0800d000 tx=0x7f3a08008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.832+0000 7f3a03fff700 1 -- 172.21.15.117:0/3929797100 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3a0c040fa0 con 0x7f3a14143c90 2020-11-20T13:56:16.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.983+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3a14057050 con 0x7f3a14143c90 2020-11-20T13:56:16.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.983+0000 7f3a03fff700 1 -- 172.21.15.117:0/3929797100 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3a0c0445c0 con 0x7f3a14143c90 2020-11-20T13:56:16.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39f00513c0 msgr2=0x7f39f0053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39f00513c0 0x7f39f0053880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f3a0800d000 tx=0x7f3a08008040).stop 2020-11-20T13:56:16.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a14143c90 msgr2=0x7f3a14148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:16.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a14143c90 0x7f3a14148740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f3a0c000f00 tx=0x7f3a0c007cb0).stop 2020-11-20T13:56:16.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 shutdown_connections 2020-11-20T13:56:16.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f39f00513c0 0x7f39f0053880 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3a1409a110 0x7f3a14140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a14143c90 0x7f3a14148740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 --2- 172.21.15.117:0/3929797100 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3a1409bd80 0x7f3a14156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:16.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 >> 172.21.15.117:0/3929797100 conn(0x7f3a140b9770 msgr2=0x7f3a1409f070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:16.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 shutdown_connections 2020-11-20T13:56:16.990 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:16.985+0000 7f3a1957f700 1 -- 172.21.15.117:0/3929797100 wait complete. 2020-11-20T13:56:16.990 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:16.996 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:17.105 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.104+0000 7fa7683b9700 1 Processor -- start 2020-11-20T13:56:17.106 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.104+0000 7fa7683b9700 1 -- start start 2020-11-20T13:56:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa7683b9700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa76009b130 0x7fa7600630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa7683b9700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7600b4860 con 0x7fa760098d60 2020-11-20T13:56:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa7683b9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7600b4b00 con 0x7fa7600635b0 2020-11-20T13:56:17.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa7683b9700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7601439f0 con 0x7fa76009b130 2020-11-20T13:56:17.109 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa765954700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa76009b130 0x7fa7600630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.109 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa766956700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa7600635b0 0x7fa76009c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60328/0 (socket says 172.21.15.117:60328) 2020-11-20T13:56:17.109 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa766956700 1 -- 172.21.15.117:0/3476130143 learned_addr learned my addr 172.21.15.117:0/3476130143 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:17.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa765954700 1 -- 172.21.15.117:0/3476130143 >> v1:172.21.15.117:6789/0 conn(0x7fa7600635b0 legacy=0x7fa76009c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa765954700 1 -- 172.21.15.117:0/3476130143 >> v1:172.21.15.85:6789/0 conn(0x7fa760098d60 legacy=0x7fa7600b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.105+0000 7fa765954700 1 -- 172.21.15.117:0/3476130143 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7600b4da0 con 0x7fa76009b130 2020-11-20T13:56:17.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa765954700 1 --2- 172.21.15.117:0/3476130143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76009b130 0x7fa7600630c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fa7600602e0 tx=0x7fa75c006e50).ready entity=mon.1 client_cookie=376aa04514d65f16 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa765153700 1 -- 172.21.15.117:0/3476130143 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa75c014070 con 0x7fa76009b130 2020-11-20T13:56:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa765153700 1 -- 172.21.15.117:0/3476130143 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c00aae0 con 0x7fa76009b130 2020-11-20T13:56:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa765153700 1 -- 172.21.15.117:0/3476130143 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa75c00ae40 con 0x7fa76009b130 2020-11-20T13:56:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa7683b9700 1 -- 172.21.15.117:0/3476130143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76009b130 msgr2=0x7fa7600630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa7683b9700 1 --2- 172.21.15.117:0/3476130143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76009b130 0x7fa7600630c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fa7600602e0 tx=0x7fa75c006e50).stop 2020-11-20T13:56:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa7683b9700 1 -- 172.21.15.117:0/3476130143 shutdown_connections 2020-11-20T13:56:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa7683b9700 1 --2- 172.21.15.117:0/3476130143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa76009b130 0x7fa7600630c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa7683b9700 1 -- 172.21.15.117:0/3476130143 >> 172.21.15.117:0/3476130143 conn(0x7fa7600b9770 msgr2=0x7fa7600b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa7683b9700 1 -- 172.21.15.117:0/3476130143 shutdown_connections 2020-11-20T13:56:17.113 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.106+0000 7fa7683b9700 1 -- 172.21.15.117:0/3476130143 wait complete. 2020-11-20T13:56:17.115 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.107+0000 7fa7683b9700 1 Processor -- start 2020-11-20T13:56:17.116 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.107+0000 7fa7683b9700 1 -- start start 2020-11-20T13:56:17.116 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa7683b9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7600635b0 0x7fa760158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.116 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa7683b9700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa760098d60 0x7fa7600b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.117 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa7683b9700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa76009b130 0x7fa760067580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.117 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa7683b9700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601588b0 con 0x7fa760098d60 2020-11-20T13:56:17.117 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa7683b9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa760142170 con 0x7fa7600635b0 2020-11-20T13:56:17.117 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa7683b9700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76013d0c0 con 0x7fa76009b130 2020-11-20T13:56:17.118 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7600635b0 0x7fa760158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.118 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7600635b0 0x7fa760158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41172/0 (socket says 172.21.15.117:41172) 2020-11-20T13:56:17.118 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 -- 172.21.15.117:0/4240582969 learned_addr learned my addr 172.21.15.117:0/4240582969 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:17.118 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 -- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa76009b130 msgr2=0x7fa760067580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.118 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766956700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa76009b130 0x7fa760067580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.119 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa765954700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa760098d60 0x7fa7600b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.119 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa76009b130 0x7fa760067580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.119 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 -- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa760098d60 msgr2=0x7fa7600b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.119 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa760098d60 0x7fa7600b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 -- 172.21.15.117:0/4240582969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76013d360 con 0x7fa7600635b0 2020-11-20T13:56:17.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa766155700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7600635b0 0x7fa760158300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa750001f50 tx=0x7fa750007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.108+0000 7fa74f7fe700 1 -- 172.21.15.117:0/4240582969 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa75000e4a0 con 0x7fa7600635b0 2020-11-20T13:56:17.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.109+0000 7fa74f7fe700 1 -- 172.21.15.117:0/4240582969 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75000e600 con 0x7fa7600635b0 2020-11-20T13:56:17.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.109+0000 7fa74f7fe700 1 -- 172.21.15.117:0/4240582969 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa75000e960 con 0x7fa7600635b0 2020-11-20T13:56:17.121 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.109+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7600b0830 con 0x7fa7600635b0 2020-11-20T13:56:17.121 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.109+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7600b0d20 con 0x7fa7600635b0 2020-11-20T13:56:17.121 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.110+0000 7fa74f7fe700 1 -- 172.21.15.117:0/4240582969 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fa750019030 con 0x7fa7600635b0 2020-11-20T13:56:17.121 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.110+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa760057050 con 0x7fa7600635b0 2020-11-20T13:56:17.122 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.110+0000 7fa74f7fe700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa7400513c0 0x7fa740053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.122 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.110+0000 7fa74f7fe700 1 -- 172.21.15.117:0/4240582969 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa7500686b0 con 0x7fa7600635b0 2020-11-20T13:56:17.122 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.110+0000 7fa765954700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa7400513c0 0x7fa740053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.122 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.110+0000 7fa765954700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa7400513c0 0x7fa740053880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fa75c017000 tx=0x7fa75c006e50).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.122 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.114+0000 7fa74f7fe700 1 -- 172.21.15.117:0/4240582969 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fa7500407f0 con 0x7fa7600635b0 2020-11-20T13:56:17.264 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.263+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa76013d4f0 con 0x7fa7600635b0 2020-11-20T13:56:17.265 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.263+0000 7fa74f7fe700 1 -- 172.21.15.117:0/4240582969 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa750043e10 con 0x7fa7600635b0 2020-11-20T13:56:17.265 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:17.266 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa7400513c0 msgr2=0x7fa740053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.267 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa7400513c0 0x7fa740053880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fa75c017000 tx=0x7fa75c006e50).stop 2020-11-20T13:56:17.267 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7600635b0 msgr2=0x7fa760158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.267 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7600635b0 0x7fa760158300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa750001f50 tx=0x7fa750007b70).stop 2020-11-20T13:56:17.267 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 shutdown_connections 2020-11-20T13:56:17.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fa7400513c0 0x7fa740053880 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa76009b130 0x7fa760067580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7600635b0 0x7fa760158300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 --2- 172.21.15.117:0/4240582969 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa760098d60 0x7fa7600b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.268 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 >> 172.21.15.117:0/4240582969 conn(0x7fa7600b9770 msgr2=0x7fa7600b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.269 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 shutdown_connections 2020-11-20T13:56:17.269 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.265+0000 7fa7683b9700 1 -- 172.21.15.117:0/4240582969 wait complete. 2020-11-20T13:56:17.277 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:17.385 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.384+0000 7f82b3784700 1 Processor -- start 2020-11-20T13:56:17.386 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.384+0000 7f82b3784700 1 -- start start 2020-11-20T13:56:17.392 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b3784700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 0x7f82ac060010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.392 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b3784700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f82ac09d800 0x7f82ac066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b3784700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ac144f90 con 0x7f82ac0be2a0 2020-11-20T13:56:17.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b3784700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac0a0360 con 0x7f82ac09d800 2020-11-20T13:56:17.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b3784700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0a0600 con 0x7f82ac144bb0 2020-11-20T13:56:17.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b1d21700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f82ac0be2a0 0x7f82ac1479e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60344/0 (socket says 172.21.15.117:60344) 2020-11-20T13:56:17.393 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b1d21700 1 -- 172.21.15.117:0/907888672 learned_addr learned my addr 172.21.15.117:0/907888672 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:17.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934473807 0 0) 0x7f82ac144f90 con 0x7f82ac0be2a0 2020-11-20T13:56:17.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8294003230 con 0x7f82ac0be2a0 2020-11-20T13:56:17.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b1520700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 0x7f82ac060010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.385+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/907888672 >> v2:172.21.15.85:3300/0 conn(0x7f82ac09d800 0x7f82ac066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.394 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1632785613 0 0) 0x7f8294003230 con 0x7f82ac0be2a0 2020-11-20T13:56:17.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 msgr2=0x7f82ac060010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 0x7f82ac060010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 >> v2:172.21.15.85:3300/0 conn(0x7f82ac09d800 msgr2=0x7f82ac066bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 --2- 172.21.15.117:0/907888672 >> v2:172.21.15.85:3300/0 conn(0x7f82ac09d800 0x7f82ac066bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.395 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac1474d0 con 0x7f82ac0be2a0 2020-11-20T13:56:17.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f82a4001a50 con 0x7f82ac0be2a0 2020-11-20T13:56:17.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 >> v1:172.21.15.117:6789/0 conn(0x7f82ac0be2a0 legacy=0x7f82ac1479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8294003720 0x7f8294003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940044d0 0x7f8294004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.396 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 0x7f8294004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0a0360 con 0x7f8294003720 2020-11-20T13:56:17.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0a0600 con 0x7f82940044d0 2020-11-20T13:56:17.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8294004260 con 0x7f82ac144bb0 2020-11-20T13:56:17.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940044d0 0x7f8294004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.397 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b1520700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8294003720 0x7f8294003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.398 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b1d21700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 0x7f8294004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.398 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b0d1f700 1 -- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 msgr2=0x7f8294004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.398 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac144bb0 0x7f8294004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.398 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b0d1f700 1 -- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8294003720 msgr2=0x7f8294003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.398 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8294003720 0x7f8294003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.399 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b0d1f700 1 -- 172.21.15.117:0/907888672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294003230 con 0x7f82940044d0 2020-11-20T13:56:17.399 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940044d0 0x7f8294004950 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f82a800c000 tx=0x7f82a8008040).ready entity=mon.1 client_cookie=2a9ef335a8675cfb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.399 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82a800e8a0 con 0x7f82940044d0 2020-11-20T13:56:17.399 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800ea00 con 0x7f82940044d0 2020-11-20T13:56:17.399 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f829bfff700 1 -- 172.21.15.117:0/907888672 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82a800ed60 con 0x7f82940044d0 2020-11-20T13:56:17.400 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b3784700 1 -- 172.21.15.117:0/907888672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940044d0 msgr2=0x7f8294004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.400 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b3784700 1 --2- 172.21.15.117:0/907888672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940044d0 0x7f8294004950 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f82a800c000 tx=0x7f82a8008040).stop 2020-11-20T13:56:17.400 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.386+0000 7f82b1d21700 1 -- 172.21.15.117:0/907888672 reap_dead start 2020-11-20T13:56:17.400 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.387+0000 7f82b3784700 1 -- 172.21.15.117:0/907888672 shutdown_connections 2020-11-20T13:56:17.400 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.387+0000 7f82b3784700 1 -- 172.21.15.117:0/907888672 >> 172.21.15.117:0/907888672 conn(0x7f82ac05c4c0 msgr2=0x7f82ac0bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.401 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.387+0000 7f82b3784700 1 -- 172.21.15.117:0/907888672 shutdown_connections 2020-11-20T13:56:17.401 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.387+0000 7f82b3784700 1 -- 172.21.15.117:0/907888672 wait complete. 2020-11-20T13:56:17.401 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 Processor -- start 2020-11-20T13:56:17.401 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 -- start start 2020-11-20T13:56:17.401 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82ac0b7370 0x7f82ac0ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.402 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac0badc0 0x7f82ac143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.402 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82ac144310 0x7f82ac159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.402 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0bab50 con 0x7f82ac144310 2020-11-20T13:56:17.402 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0b5590 con 0x7f82ac0b7370 2020-11-20T13:56:17.402 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.388+0000 7f82b3784700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0b5830 con 0x7f82ac0badc0 2020-11-20T13:56:17.403 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82ac0b7370 0x7f82ac0ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.403 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82ac0b7370 0x7f82ac0ba4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41192/0 (socket says 172.21.15.117:41192) 2020-11-20T13:56:17.403 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 -- 172.21.15.117:0/1601495030 learned_addr learned my addr 172.21.15.117:0/1601495030 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:17.403 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1d21700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82ac144310 0x7f82ac159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.403 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac0badc0 0x7f82ac143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.404 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 -- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac0badc0 msgr2=0x7f82ac143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.404 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac0badc0 0x7f82ac143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.404 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 -- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82ac144310 msgr2=0x7f82ac159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.404 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82ac144310 0x7f82ac159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.404 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 -- 172.21.15.117:0/1601495030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac0b7750 con 0x7f82ac0b7370 2020-11-20T13:56:17.405 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b1520700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82ac0b7370 0x7f82ac0ba4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f829c001e60 tx=0x7f829c006d60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.405 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f829a7fc700 1 -- 172.21.15.117:0/1601495030 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f829c00f6e0 con 0x7f82ac0b7370 2020-11-20T13:56:17.405 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f829a7fc700 1 -- 172.21.15.117:0/1601495030 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c00f840 con 0x7f82ac0b7370 2020-11-20T13:56:17.405 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac144770 con 0x7f82ac0b7370 2020-11-20T13:56:17.405 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f829a7fc700 1 -- 172.21.15.117:0/1601495030 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f829c00f9b0 con 0x7f82ac0b7370 2020-11-20T13:56:17.406 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.389+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac159a60 con 0x7f82ac0b7370 2020-11-20T13:56:17.406 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.390+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac0ba9a0 con 0x7f82ac0b7370 2020-11-20T13:56:17.406 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.390+0000 7f829a7fc700 1 -- 172.21.15.117:0/1601495030 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f829c024070 con 0x7f82ac0b7370 2020-11-20T13:56:17.406 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.391+0000 7f829a7fc700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f828c051320 0x7f828c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.406 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.391+0000 7f829a7fc700 1 -- 172.21.15.117:0/1601495030 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f829c0696f0 con 0x7f82ac0b7370 2020-11-20T13:56:17.407 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.391+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f828c051320 0x7f828c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.407 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.391+0000 7f82b0d1f700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f828c051320 0x7f828c0537e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f82a8008000 tx=0x7f82a801a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.407 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.394+0000 7f829a7fc700 1 -- 172.21.15.117:0/1601495030 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f829c041910 con 0x7f82ac0b7370 2020-11-20T13:56:17.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.546+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82ac057050 con 0x7f82ac0b7370 2020-11-20T13:56:17.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.546+0000 7f829a7fc700 1 -- 172.21.15.117:0/1601495030 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7f829c044f30 con 0x7f82ac0b7370 2020-11-20T13:56:17.548 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:17.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f828c051320 msgr2=0x7f828c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f828c051320 0x7f828c0537e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f82a8008000 tx=0x7f82a801a000).stop 2020-11-20T13:56:17.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82ac0b7370 msgr2=0x7f82ac0ba4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82ac0b7370 0x7f82ac0ba4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f829c001e60 tx=0x7f829c006d60).stop 2020-11-20T13:56:17.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 shutdown_connections 2020-11-20T13:56:17.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f828c051320 0x7f828c0537e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f82ac144310 0x7f82ac159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82ac0b7370 0x7f82ac0ba4b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 --2- 172.21.15.117:0/1601495030 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f82ac0badc0 0x7f82ac143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 >> 172.21.15.117:0/1601495030 conn(0x7f82ac05c4c0 msgr2=0x7f82ac144f90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 shutdown_connections 2020-11-20T13:56:17.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.548+0000 7f82b3784700 1 -- 172.21.15.117:0/1601495030 wait complete. 2020-11-20T13:56:17.560 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:17.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fcda65700 1 Processor -- start 2020-11-20T13:56:17.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fcda65700 1 -- start start 2020-11-20T13:56:17.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fcda65700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2fc809bd80 0x7f2fc809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fcda65700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc80b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fcda65700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc80999d0 con 0x7f2fc809a110 2020-11-20T13:56:17.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fcda65700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc8099c70 con 0x7f2fc809bd80 2020-11-20T13:56:17.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fcda65700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc809c160 con 0x7f2fc8143eb0 2020-11-20T13:56:17.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fc77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2fc809bd80 0x7f2fc809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fc6ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2fc809a110 0x7f2fc81437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60362/0 (socket says 172.21.15.117:60362) 2020-11-20T13:56:17.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fc6ffd700 1 -- 172.21.15.117:0/2430452653 learned_addr learned my addr 172.21.15.117:0/2430452653 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:17.675 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.670+0000 7f2fc77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2fc809bd80 0x7f2fc809cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41200/0 (socket says 172.21.15.117:41200) 2020-11-20T13:56:17.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc7fff700 1 --2- 172.21.15.117:0/2430452653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc80b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc77fe700 1 -- 172.21.15.117:0/2430452653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 msgr2=0x7f2fc80b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc77fe700 1 --2- 172.21.15.117:0/2430452653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc80b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc77fe700 1 -- 172.21.15.117:0/2430452653 >> v1:172.21.15.117:6789/0 conn(0x7f2fc809a110 legacy=0x7f2fc81437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.676 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc67fc700 1 -- 172.21.15.117:0/2430452653 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134576519 0 0) 0x7f2fc80999d0 con 0x7f2fc809a110 2020-11-20T13:56:17.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc77fe700 1 -- 172.21.15.117:0/2430452653 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc809a5c0 con 0x7f2fc809bd80 2020-11-20T13:56:17.677 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc7fff700 1 --2- 172.21.15.117:0/2430452653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:17.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc77fe700 1 --2- 172.21.15.117:0/2430452653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc809cb40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f2fb8002f30 tx=0x7f2fb8006dc0).ready entity=mon.1 client_cookie=2b07bd956640fd27 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc67fc700 1 -- 172.21.15.117:0/2430452653 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb80097f0 con 0x7f2fc809bd80 2020-11-20T13:56:17.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc67fc700 1 -- 172.21.15.117:0/2430452653 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb8009950 con 0x7f2fc809bd80 2020-11-20T13:56:17.678 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fc67fc700 1 -- 172.21.15.117:0/2430452653 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fb8009cb0 con 0x7f2fc809bd80 2020-11-20T13:56:17.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fcda65700 1 -- 172.21.15.117:0/2430452653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 msgr2=0x7f2fc809cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fcda65700 1 --2- 172.21.15.117:0/2430452653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc809cb40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f2fb8002f30 tx=0x7f2fb8006dc0).stop 2020-11-20T13:56:17.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fcda65700 1 -- 172.21.15.117:0/2430452653 shutdown_connections 2020-11-20T13:56:17.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fcda65700 1 --2- 172.21.15.117:0/2430452653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.679 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fcda65700 1 --2- 172.21.15.117:0/2430452653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc809cb40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fcda65700 1 -- 172.21.15.117:0/2430452653 >> 172.21.15.117:0/2430452653 conn(0x7f2fc80b9770 msgr2=0x7f2fc80b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.671+0000 7f2fcda65700 1 -- 172.21.15.117:0/2430452653 shutdown_connections 2020-11-20T13:56:17.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.672+0000 7f2fcda65700 1 -- 172.21.15.117:0/2430452653 wait complete. 2020-11-20T13:56:17.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.672+0000 7f2fcda65700 1 Processor -- start 2020-11-20T13:56:17.680 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fcda65700 1 -- start start 2020-11-20T13:56:17.683 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fcda65700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2fc809a110 0x7f2fc80b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.683 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fcda65700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc8145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.683 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fcda65700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc8145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.683 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fcda65700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc8156210 con 0x7f2fc809a110 2020-11-20T13:56:17.684 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fcda65700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc8067440 con 0x7f2fc809bd80 2020-11-20T13:56:17.684 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fcda65700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc8156400 con 0x7f2fc8143eb0 2020-11-20T13:56:17.684 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fc6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc8145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.684 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fc6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc8145120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41208/0 (socket says 172.21.15.117:41208) 2020-11-20T13:56:17.684 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.673+0000 7f2fc6ffd700 1 -- 172.21.15.117:0/3614262933 learned_addr learned my addr 172.21.15.117:0/3614262933 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc7fff700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc8145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc77fe700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2fc809a110 0x7f2fc80b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc6ffd700 1 -- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 msgr2=0x7f2fc8145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.685 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc6ffd700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc8145610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc6ffd700 1 -- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2fc809a110 msgr2=0x7f2fc80b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc6ffd700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2fc809a110 0x7f2fc80b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc6ffd700 1 -- 172.21.15.117:0/3614262933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc81461f0 con 0x7f2fc809bd80 2020-11-20T13:56:17.686 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc7fff700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc8145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc77fe700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2fc809a110 0x7f2fc80b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc6ffd700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc8145120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f2fbc000e90 tx=0x7f2fbc0079f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.688 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc4ff9700 1 -- 172.21.15.117:0/3614262933 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fbc007320 con 0x7f2fc809bd80 2020-11-20T13:56:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc4ff9700 1 -- 172.21.15.117:0/3614262933 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc007480 con 0x7f2fc809bd80 2020-11-20T13:56:17.689 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc8066b70 con 0x7f2fc809bd80 2020-11-20T13:56:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fc4ff9700 1 -- 172.21.15.117:0/3614262933 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2fbc0057d0 con 0x7f2fc809bd80 2020-11-20T13:56:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.674+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc80671e0 con 0x7f2fc809bd80 2020-11-20T13:56:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.675+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc81565c0 con 0x7f2fc809bd80 2020-11-20T13:56:17.690 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.675+0000 7f2fc4ff9700 1 -- 172.21.15.117:0/3614262933 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2fbc010030 con 0x7f2fc809bd80 2020-11-20T13:56:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.675+0000 7f2fc4ff9700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2fa80513c0 0x7f2fa8053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.675+0000 7f2fc4ff9700 1 -- 172.21.15.117:0/3614262933 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f2fbc068e00 con 0x7f2fc809bd80 2020-11-20T13:56:17.691 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.675+0000 7f2fc77fe700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2fa80513c0 0x7f2fa8053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.692 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.676+0000 7f2fc77fe700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2fa80513c0 0x7f2fa8053880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2fb8016000 tx=0x7f2fb8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.692 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.679+0000 7f2fc4ff9700 1 -- 172.21.15.117:0/3614262933 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2fbc041020 con 0x7f2fc809bd80 2020-11-20T13:56:17.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.828+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2fc8057050 con 0x7f2fc809bd80 2020-11-20T13:56:17.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.829+0000 7f2fc4ff9700 1 -- 172.21.15.117:0/3614262933 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2fbc044640 con 0x7f2fc809bd80 2020-11-20T13:56:17.830 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:17.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2fa80513c0 msgr2=0x7f2fa8053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2fa80513c0 0x7f2fa8053880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f2fb8016000 tx=0x7f2fb8015000).stop 2020-11-20T13:56:17.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 msgr2=0x7f2fc8145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc8145120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f2fbc000e90 tx=0x7f2fbc0079f0).stop 2020-11-20T13:56:17.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 shutdown_connections 2020-11-20T13:56:17.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2fa80513c0 0x7f2fa8053880 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.832 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2fc8143eb0 0x7f2fc8145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2fc809a110 0x7f2fc80b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 --2- 172.21.15.117:0/3614262933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fc809bd80 0x7f2fc8145120 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 >> 172.21.15.117:0/3614262933 conn(0x7f2fc80b9770 msgr2=0x7f2fc8142500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 shutdown_connections 2020-11-20T13:56:17.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.830+0000 7f2fcda65700 1 -- 172.21.15.117:0/3614262933 wait complete. 2020-11-20T13:56:17.842 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.947+0000 7f1607896700 1 Processor -- start 2020-11-20T13:56:17.949 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.947+0000 7f1607896700 1 -- start start 2020-11-20T13:56:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1607896700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f16001438e0 0x7f1600143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1607896700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160009b920 con 0x7f16000ae6f0 2020-11-20T13:56:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1607896700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16000b5510 con 0x7f1600098e30 2020-11-20T13:56:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1607896700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1600144230 con 0x7f16001438e0 2020-11-20T13:56:17.951 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1604e31700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f16001438e0 0x7f1600143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1605632700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1600098e30 0x7f160009b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60374/0 (socket says 172.21.15.117:60374) 2020-11-20T13:56:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1605632700 1 -- 172.21.15.117:0/2241270243 learned_addr learned my addr 172.21.15.117:0/2241270243 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1604e31700 1 -- 172.21.15.117:0/2241270243 >> v1:172.21.15.117:6789/0 conn(0x7f1600098e30 legacy=0x7f160009b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.952 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1604e31700 1 -- 172.21.15.117:0/2241270243 >> v1:172.21.15.85:6789/0 conn(0x7f16000ae6f0 legacy=0x7f16000aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:17.953 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.948+0000 7f1604e31700 1 -- 172.21.15.117:0/2241270243 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16000b57b0 con 0x7f16001438e0 2020-11-20T13:56:17.954 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1604e31700 1 --2- 172.21.15.117:0/2241270243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16001438e0 0x7f1600143d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f16000b5dd0 tx=0x7f15fc006dc0).ready entity=mon.1 client_cookie=f10a8a5711da4b88 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.954 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f15effff700 1 -- 172.21.15.117:0/2241270243 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15fc0096a0 con 0x7f16001438e0 2020-11-20T13:56:17.954 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f15effff700 1 -- 172.21.15.117:0/2241270243 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15fc009800 con 0x7f16001438e0 2020-11-20T13:56:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f15effff700 1 -- 172.21.15.117:0/2241270243 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15fc009b60 con 0x7f16001438e0 2020-11-20T13:56:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1607896700 1 -- 172.21.15.117:0/2241270243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16001438e0 msgr2=0x7f1600143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1607896700 1 --2- 172.21.15.117:0/2241270243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16001438e0 0x7f1600143d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f16000b5dd0 tx=0x7f15fc006dc0).stop 2020-11-20T13:56:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1607896700 1 -- 172.21.15.117:0/2241270243 shutdown_connections 2020-11-20T13:56:17.955 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1607896700 1 --2- 172.21.15.117:0/2241270243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16001438e0 0x7f1600143d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.956 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1607896700 1 -- 172.21.15.117:0/2241270243 >> 172.21.15.117:0/2241270243 conn(0x7f160005b9f0 msgr2=0x7f160005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:17.956 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1607896700 1 -- 172.21.15.117:0/2241270243 shutdown_connections 2020-11-20T13:56:17.956 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.949+0000 7f1607896700 1 -- 172.21.15.117:0/2241270243 wait complete. 2020-11-20T13:56:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.950+0000 7f1607896700 1 Processor -- start 2020-11-20T13:56:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.950+0000 7f1607896700 1 -- start start 2020-11-20T13:56:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1607896700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1600098e30 0x7f160009ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.959 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1607896700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f16000ae6f0 0x7f160015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1607896700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f16001438e0 0x7f1600066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1607896700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160009e680 con 0x7f16001438e0 2020-11-20T13:56:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1607896700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160009f220 con 0x7f1600098e30 2020-11-20T13:56:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1607896700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160009f4c0 con 0x7f16000ae6f0 2020-11-20T13:56:17.960 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1600098e30 0x7f160009ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1600098e30 0x7f160009ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41218/0 (socket says 172.21.15.117:41218) 2020-11-20T13:56:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 -- 172.21.15.117:0/3766848631 learned_addr learned my addr 172.21.15.117:0/3766848631 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1604e31700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f16000ae6f0 0x7f160015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605e33700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f16001438e0 0x7f1600066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.961 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 -- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f16000ae6f0 msgr2=0x7f160015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.962 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f16000ae6f0 0x7f160015a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.963 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 -- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f16001438e0 msgr2=0x7f1600066f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:17.963 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f16001438e0 0x7f1600066f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:17.963 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605632700 1 -- 172.21.15.117:0/3766848631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600156640 con 0x7f1600098e30 2020-11-20T13:56:17.964 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1604e31700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f16000ae6f0 0x7f160015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:17.965 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.951+0000 7f1605e33700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f16001438e0 0x7f1600066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:17.965 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.952+0000 7f1605632700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1600098e30 0x7f160009ebf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f15f0001190 tx=0x7f15f0007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.965 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.952+0000 7f15ee7fc700 1 -- 172.21.15.117:0/3766848631 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15f0007320 con 0x7f1600098e30 2020-11-20T13:56:17.965 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.952+0000 7f15ee7fc700 1 -- 172.21.15.117:0/3766848631 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f0007480 con 0x7f1600098e30 2020-11-20T13:56:17.966 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.952+0000 7f15ee7fc700 1 -- 172.21.15.117:0/3766848631 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15f00057d0 con 0x7f1600098e30 2020-11-20T13:56:17.966 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.952+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160009d4c0 con 0x7f1600098e30 2020-11-20T13:56:17.966 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.952+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160009da10 con 0x7f1600098e30 2020-11-20T13:56:17.966 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.953+0000 7f15ee7fc700 1 -- 172.21.15.117:0/3766848631 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f15f0010030 con 0x7f1600098e30 2020-11-20T13:56:17.966 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.953+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1600057050 con 0x7f1600098e30 2020-11-20T13:56:17.967 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.953+0000 7f15ee7fc700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f15e0051210 0x7f15e00536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:17.967 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.953+0000 7f15ee7fc700 1 -- 172.21.15.117:0/3766848631 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f15f0068f90 con 0x7f1600098e30 2020-11-20T13:56:17.967 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.953+0000 7f1604e31700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f15e0051210 0x7f15e00536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:17.967 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.953+0000 7f1604e31700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f15e0051210 0x7f15e00536d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f15fc017000 tx=0x7f15fc015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:17.968 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:17.956+0000 7f15ee7fc700 1 -- 172.21.15.117:0/3766848631 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f15f00411b0 con 0x7f1600098e30 2020-11-20T13:56:18.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.106+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1600067b20 con 0x7f1600098e30 2020-11-20T13:56:18.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.107+0000 7f15ee7fc700 1 -- 172.21.15.117:0/3766848631 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16347 (secure 0 0 0) 0x7f15f00447d0 con 0x7f1600098e30 2020-11-20T13:56:18.109 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:18.109 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":25,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:55:51.676406+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:55:45.628162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T13:55:48.335298+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:18.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f15e0051210 msgr2=0x7f15e00536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f15e0051210 0x7f15e00536d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f15fc017000 tx=0x7f15fc015000).stop 2020-11-20T13:56:18.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1600098e30 msgr2=0x7f160009ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1600098e30 0x7f160009ebf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f15f0001190 tx=0x7f15f0007b50).stop 2020-11-20T13:56:18.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 shutdown_connections 2020-11-20T13:56:18.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f15e0051210 0x7f15e00536d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.111 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f16001438e0 0x7f1600066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1600098e30 0x7f160009ebf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 --2- 172.21.15.117:0/3766848631 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f16000ae6f0 0x7f160015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 >> 172.21.15.117:0/3766848631 conn(0x7f160005b9f0 msgr2=0x7f16000b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.108+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 shutdown_connections 2020-11-20T13:56:18.112 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.109+0000 7f1607896700 1 -- 172.21.15.117:0/3766848631 wait complete. 2020-11-20T13:56:18.120 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:55:45.628162+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:55:48.335298+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:18.121 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:18.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.228+0000 7fc5c37c6700 1 Processor -- start 2020-11-20T13:56:18.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.228+0000 7fc5c37c6700 1 -- start start 2020-11-20T13:56:18.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5c37c6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 0x7fc5bc064d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5c37c6700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc147660 con 0x7fc5bc064960 2020-11-20T13:56:18.233 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5c37c6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc060bc0 con 0x7fc5bc0654e0 2020-11-20T13:56:18.234 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5c37c6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc060e60 con 0x7fc5bc1470f0 2020-11-20T13:56:18.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5c1d63700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc5bc0654e0 0x7fc5bc147ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60386/0 (socket says 172.21.15.117:60386) 2020-11-20T13:56:18.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5c1d63700 1 -- 172.21.15.117:0/2753272644 learned_addr learned my addr 172.21.15.117:0/2753272644 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:18.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5c0d61700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 0x7fc5bc064d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719493138 0 0) 0x7fc5bc060bc0 con 0x7fc5bc0654e0 2020-11-20T13:56:18.235 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a4003230 con 0x7fc5bc0654e0 2020-11-20T13:56:18.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.229+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3943318277 0 0) 0x7fc5a4003230 con 0x7fc5bc0654e0 2020-11-20T13:56:18.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 msgr2=0x7fc5bc064d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:18.236 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 0x7fc5bc064d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 >> v1:172.21.15.85:6789/0 conn(0x7fc5bc064960 legacy=0x7fc5bc05f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc05fb80 con 0x7fc5bc0654e0 2020-11-20T13:56:18.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fc5b4001a50 con 0x7fc5bc0654e0 2020-11-20T13:56:18.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 >> v1:172.21.15.117:6789/0 conn(0x7fc5bc0654e0 legacy=0x7fc5bc147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.237 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4003720 0x7fc5a4003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 0x7fc5a40044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5a40049c0 0x7fc5a4004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc060e60 con 0x7fc5a40049c0 2020-11-20T13:56:18.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4004130 con 0x7fc5a4003720 2020-11-20T13:56:18.238 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40042f0 con 0x7fc5bc1470f0 2020-11-20T13:56:18.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1562700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4003720 0x7fc5a4003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c0d61700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 0x7fc5a40044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1d63700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5a40049c0 0x7fc5a4004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1562700 1 -- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 msgr2=0x7fc5a40044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:18.239 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1562700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 0x7fc5a40044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1562700 1 -- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5a40049c0 msgr2=0x7fc5a4004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:18.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1562700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5a40049c0 0x7fc5a4004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1562700 1 -- 172.21.15.117:0/2753272644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4003230 con 0x7fc5a4003720 2020-11-20T13:56:18.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c0d61700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc1470f0 0x7fc5a40044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:18.240 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1562700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4003720 0x7fc5a4003b00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fc5ac000fe0 tx=0x7fc5ac007ab0).ready entity=mon.1 client_cookie=17ff2cbb83a73af7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ac00c870 con 0x7fc5a4003720 2020-11-20T13:56:18.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac00c9d0 con 0x7fc5a4003720 2020-11-20T13:56:18.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5abfff700 1 -- 172.21.15.117:0/2753272644 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ac00cd30 con 0x7fc5a4003720 2020-11-20T13:56:18.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c37c6700 1 -- 172.21.15.117:0/2753272644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4003720 msgr2=0x7fc5a4003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.241 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c37c6700 1 --2- 172.21.15.117:0/2753272644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4003720 0x7fc5a4003b00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fc5ac000fe0 tx=0x7fc5ac007ab0).stop 2020-11-20T13:56:18.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.230+0000 7fc5c1d63700 1 -- 172.21.15.117:0/2753272644 reap_dead start 2020-11-20T13:56:18.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.231+0000 7fc5c37c6700 1 -- 172.21.15.117:0/2753272644 shutdown_connections 2020-11-20T13:56:18.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.231+0000 7fc5c37c6700 1 -- 172.21.15.117:0/2753272644 >> 172.21.15.117:0/2753272644 conn(0x7fc5bc0bb980 msgr2=0x7fc5bc0bbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.231+0000 7fc5c37c6700 1 -- 172.21.15.117:0/2753272644 shutdown_connections 2020-11-20T13:56:18.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.231+0000 7fc5c37c6700 1 -- 172.21.15.117:0/2753272644 wait complete. 2020-11-20T13:56:18.242 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 Processor -- start 2020-11-20T13:56:18.243 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 -- start start 2020-11-20T13:56:18.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc0b50c0 0x7fc5bc09b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5bc144b60 0x7fc5bc144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.246 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc15a150 con 0x7fc5bc144b60 2020-11-20T13:56:18.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc09b700 con 0x7fc5bc1597f0 2020-11-20T13:56:18.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c37c6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b58f0 con 0x7fc5bc0b50c0 2020-11-20T13:56:18.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c1562700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c1562700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41238/0 (socket says 172.21.15.117:41238) 2020-11-20T13:56:18.247 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c1562700 1 -- 172.21.15.117:0/1464404549 learned_addr learned my addr 172.21.15.117:0/1464404549 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:18.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c0d61700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc0b50c0 0x7fc5bc09b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.232+0000 7fc5c1d63700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5bc144b60 0x7fc5bc144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c1562700 1 -- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc0b50c0 msgr2=0x7fc5bc09b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c1562700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc0b50c0 0x7fc5bc09b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.248 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c1562700 1 -- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5bc144b60 msgr2=0x7fc5bc144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.249 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c1562700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5bc144b60 0x7fc5bc144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.249 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c1562700 1 -- 172.21.15.117:0/1464404549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0b5b60 con 0x7fc5bc1597f0 2020-11-20T13:56:18.249 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c1562700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc159bd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fc5bc0b5a80 tx=0x7fc5ac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5aa7fc700 1 -- 172.21.15.117:0/1464404549 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ac013e30 con 0x7fc5bc1597f0 2020-11-20T13:56:18.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5aa7fc700 1 -- 172.21.15.117:0/1464404549 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac007030 con 0x7fc5bc1597f0 2020-11-20T13:56:18.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc0a03b0 con 0x7fc5bc1597f0 2020-11-20T13:56:18.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5aa7fc700 1 -- 172.21.15.117:0/1464404549 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5ac007390 con 0x7fc5bc1597f0 2020-11-20T13:56:18.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.233+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc0a0850 con 0x7fc5bc1597f0 2020-11-20T13:56:18.250 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.234+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc0a0540 con 0x7fc5bc1597f0 2020-11-20T13:56:18.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.234+0000 7fc5aa7fc700 1 -- 172.21.15.117:0/1464404549 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc5ac024030 con 0x7fc5bc1597f0 2020-11-20T13:56:18.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.235+0000 7fc5aa7fc700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc59c051290 0x7fc59c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.235+0000 7fc5aa7fc700 1 -- 172.21.15.117:0/1464404549 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fc5ac069ed0 con 0x7fc5bc1597f0 2020-11-20T13:56:18.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.235+0000 7fc5c0d61700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc59c051290 0x7fc59c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.251 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.235+0000 7fc5c0d61700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc59c051290 0x7fc59c053750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc5b8008000 tx=0x7fc5b8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.252 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.238+0000 7fc5aa7fc700 1 -- 172.21.15.117:0/1464404549 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc5ac0421e0 con 0x7fc5bc1597f0 2020-11-20T13:56:18.387 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.386+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc5bc057050 con 0x7fc5bc1597f0 2020-11-20T13:56:18.388 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.386+0000 7fc5aa7fc700 1 -- 172.21.15.117:0/1464404549 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v25) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc5ac022070 con 0x7fc5bc1597f0 2020-11-20T13:56:18.388 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:18.389 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc59c051290 msgr2=0x7fc59c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.389 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc59c051290 0x7fc59c053750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc5b8008000 tx=0x7fc5b8008040).stop 2020-11-20T13:56:18.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc1597f0 msgr2=0x7fc5bc159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc159bd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fc5bc0b5a80 tx=0x7fc5ac016000).stop 2020-11-20T13:56:18.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 shutdown_connections 2020-11-20T13:56:18.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc59c051290 0x7fc59c053750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.390 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc5bc144b60 0x7fc5bc144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc159bd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 --2- 172.21.15.117:0/1464404549 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc5bc0b50c0 0x7fc5bc09b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 >> 172.21.15.117:0/1464404549 conn(0x7fc5bc0bb980 msgr2=0x7fc5bc05f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 shutdown_connections 2020-11-20T13:56:18.391 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.388+0000 7fc5c37c6700 1 -- 172.21.15.117:0/1464404549 wait complete. 2020-11-20T13:56:18.400 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:18.508 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.507+0000 7fdadbd33700 1 Processor -- start 2020-11-20T13:56:18.508 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.507+0000 7fdadbd33700 1 -- start start 2020-11-20T13:56:18.510 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdadbd33700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fdad409cb40 0x7fdad41437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdadbd33700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdad4143c90 0x7fdad40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdadbd33700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40999d0 con 0x7fdad409bd80 2020-11-20T13:56:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdadbd33700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad40997f0 con 0x7fdad409cb40 2020-11-20T13:56:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdadbd33700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad4099c70 con 0x7fdad4143c90 2020-11-20T13:56:18.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdada2d0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdad4143c90 0x7fdad40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdad9acf700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fdad409bd80 0x7fdad409a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60406/0 (socket says 172.21.15.117:60406) 2020-11-20T13:56:18.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdad9acf700 1 -- 172.21.15.117:0/351380057 learned_addr learned my addr 172.21.15.117:0/351380057 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:18.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdada2d0700 1 -- 172.21.15.117:0/351380057 >> v2:172.21.15.85:3300/0 conn(0x7fdad409cb40 msgr2=0x7fdad41437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:18.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdad92ce700 1 --2- 172.21.15.117:0/351380057 >> v2:172.21.15.85:3300/0 conn(0x7fdad409cb40 0x7fdad41437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdad8acd700 1 -- 172.21.15.117:0/351380057 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253225997 0 0) 0x7fdad40999d0 con 0x7fdad409bd80 2020-11-20T13:56:18.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdada2d0700 1 --2- 172.21.15.117:0/351380057 >> v2:172.21.15.85:3300/0 conn(0x7fdad409cb40 0x7fdad41437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdada2d0700 1 -- 172.21.15.117:0/351380057 >> v1:172.21.15.117:6789/0 conn(0x7fdad409bd80 legacy=0x7fdad409a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdada2d0700 1 -- 172.21.15.117:0/351380057 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4144120 con 0x7fdad4143c90 2020-11-20T13:56:18.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.508+0000 7fdada2d0700 1 --2- 172.21.15.117:0/351380057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad4143c90 0x7fdad40b4110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fdacc0032b0 tx=0x7fdacc0091a0).ready entity=mon.1 client_cookie=4351ad76c1611367 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdad8acd700 1 -- 172.21.15.117:0/351380057 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdacc00ba50 con 0x7fdad4143c90 2020-11-20T13:56:18.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdad8acd700 1 -- 172.21.15.117:0/351380057 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc00bbb0 con 0x7fdad4143c90 2020-11-20T13:56:18.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdad8acd700 1 -- 172.21.15.117:0/351380057 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdacc00bd20 con 0x7fdad4143c90 2020-11-20T13:56:18.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 -- 172.21.15.117:0/351380057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad4143c90 msgr2=0x7fdad40b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 --2- 172.21.15.117:0/351380057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad4143c90 0x7fdad40b4110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fdacc0032b0 tx=0x7fdacc0091a0).stop 2020-11-20T13:56:18.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 -- 172.21.15.117:0/351380057 shutdown_connections 2020-11-20T13:56:18.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 --2- 172.21.15.117:0/351380057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad4143c90 0x7fdad40b4110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 --2- 172.21.15.117:0/351380057 >> v2:172.21.15.85:3300/0 conn(0x7fdad409cb40 0x7fdad41437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 -- 172.21.15.117:0/351380057 >> 172.21.15.117:0/351380057 conn(0x7fdad40b9770 msgr2=0x7fdad40b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 -- 172.21.15.117:0/351380057 shutdown_connections 2020-11-20T13:56:18.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.509+0000 7fdadbd33700 1 -- 172.21.15.117:0/351380057 wait complete. 2020-11-20T13:56:18.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.510+0000 7fdadbd33700 1 Processor -- start 2020-11-20T13:56:18.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.510+0000 7fdadbd33700 1 -- start start 2020-11-20T13:56:18.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fdad409bd80 0x7fdad4138130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad409cb40 0x7fdad412e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fdad4143c90 0x7fdad412ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.519 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4139ee0 con 0x7fdad409bd80 2020-11-20T13:56:18.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4156d60 con 0x7fdad409cb40 2020-11-20T13:56:18.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad413a0d0 con 0x7fdad4143c90 2020-11-20T13:56:18.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad409cb40 0x7fdad412e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.520 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad409cb40 0x7fdad412e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41254/0 (socket says 172.21.15.117:41254) 2020-11-20T13:56:18.521 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 -- 172.21.15.117:0/916522022 learned_addr learned my addr 172.21.15.117:0/916522022 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:18.521 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 -- 172.21.15.117:0/916522022 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fdad4143c90 msgr2=0x7fdad412ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.521 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fdad4143c90 0x7fdad412ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:18.521 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 -- 172.21.15.117:0/916522022 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fdad409bd80 msgr2=0x7fdad4138130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.521 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad9acf700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fdad409bd80 0x7fdad4138130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.522 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fdad409bd80 0x7fdad4138130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.522 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 -- 172.21.15.117:0/916522022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4156a90 con 0x7fdad409cb40 2020-11-20T13:56:18.522 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdad92ce700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad409cb40 0x7fdad412e810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fdad413a450 tx=0x7fdad0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.522 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdac2ffd700 1 -- 172.21.15.117:0/916522022 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdad0007630 con 0x7fdad409cb40 2020-11-20T13:56:18.522 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdac2ffd700 1 -- 172.21.15.117:0/916522022 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad000c030 con 0x7fdad409cb40 2020-11-20T13:56:18.523 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad4138970 con 0x7fdad409cb40 2020-11-20T13:56:18.523 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdac2ffd700 1 -- 172.21.15.117:0/916522022 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdad0007980 con 0x7fdad409cb40 2020-11-20T13:56:18.523 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.511+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad41333f0 con 0x7fdad409cb40 2020-11-20T13:56:18.523 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.513+0000 7fdac2ffd700 1 -- 172.21.15.117:0/916522022 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fdad0019030 con 0x7fdad409cb40 2020-11-20T13:56:18.523 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.513+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad413a290 con 0x7fdad409cb40 2020-11-20T13:56:18.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.513+0000 7fdac2ffd700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fdab4051210 0x7fdab40536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.513+0000 7fdac2ffd700 1 -- 172.21.15.117:0/916522022 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fdad00689f0 con 0x7fdad409cb40 2020-11-20T13:56:18.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.513+0000 7fdad9acf700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fdab4051210 0x7fdab40536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.513+0000 7fdad9acf700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fdab4051210 0x7fdab40536d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fdac4008000 tx=0x7fdac4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.516+0000 7fdac2ffd700 1 -- 172.21.15.117:0/916522022 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fdad006ae60 con 0x7fdad409cb40 2020-11-20T13:56:18.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.666+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdad4057050 con 0x7fdad409cb40 2020-11-20T13:56:18.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.666+0000 7fdac2ffd700 1 -- 172.21.15.117:0/916522022 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdad0040c60 con 0x7fdad409cb40 2020-11-20T13:56:18.668 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:18.669 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fdab4051210 msgr2=0x7fdab40536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fdab4051210 0x7fdab40536d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fdac4008000 tx=0x7fdac4009040).stop 2020-11-20T13:56:18.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad409cb40 msgr2=0x7fdad412e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad409cb40 0x7fdad412e810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fdad413a450 tx=0x7fdad0008040).stop 2020-11-20T13:56:18.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 shutdown_connections 2020-11-20T13:56:18.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fdab4051210 0x7fdab40536d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fdad4143c90 0x7fdad412ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:18.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fdad409bd80 0x7fdad4138130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 --2- 172.21.15.117:0/916522022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad409cb40 0x7fdad412e810 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 >> 172.21.15.117:0/916522022 conn(0x7fdad40b9770 msgr2=0x7fdad40b9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.672 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 shutdown_connections 2020-11-20T13:56:18.672 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.668+0000 7fdadbd33700 1 -- 172.21.15.117:0/916522022 wait complete. 2020-11-20T13:56:18.679 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:18.787 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.786+0000 7feed052a700 1 Processor -- start 2020-11-20T13:56:18.787 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.786+0000 7feed052a700 1 -- start start 2020-11-20T13:56:18.790 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.790 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feec809a110 0x7feec81437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.790 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7feec8143c90 0x7feec80b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.790 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec80999d0 con 0x7feec8143c90 2020-11-20T13:56:18.791 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec8099c70 con 0x7feec809a110 2020-11-20T13:56:18.791 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec8144120 con 0x7feec809bd80 2020-11-20T13:56:18.792 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feec809a110 0x7feec81437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.792 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feec809a110 0x7feec81437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41264/0 (socket says 172.21.15.117:41264) 2020-11-20T13:56:18.792 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 -- 172.21.15.117:0/903822614 learned_addr learned my addr 172.21.15.117:0/903822614 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:18.792 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feeceac7700 1 --2- 172.21.15.117:0/903822614 >> v2:172.21.15.85:3300/0 conn(0x7feec8143c90 0x7feec80b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.792 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feece2c6700 1 --2- 172.21.15.117:0/903822614 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.793 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 -- 172.21.15.117:0/903822614 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 msgr2=0x7feec809cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:18.793 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 --2- 172.21.15.117:0/903822614 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec809cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.793 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 -- 172.21.15.117:0/903822614 >> v2:172.21.15.85:3300/0 conn(0x7feec8143c90 msgr2=0x7feec80b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:18.793 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 --2- 172.21.15.117:0/903822614 >> v2:172.21.15.85:3300/0 conn(0x7feec8143c90 0x7feec80b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.794 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 -- 172.21.15.117:0/903822614 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec809c160 con 0x7feec809a110 2020-11-20T13:56:18.794 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feeceac7700 1 --2- 172.21.15.117:0/903822614 >> v2:172.21.15.85:3300/0 conn(0x7feec8143c90 0x7feec80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:18.794 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feece2c6700 1 --2- 172.21.15.117:0/903822614 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec809cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:18.794 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecdac5700 1 --2- 172.21.15.117:0/903822614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec809a110 0x7feec81437a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7feebc002f30 tx=0x7feebc006e80).ready entity=mon.1 client_cookie=9a15496d6ebcabbf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.795 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecd2c4700 1 -- 172.21.15.117:0/903822614 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feebc0097f0 con 0x7feec809a110 2020-11-20T13:56:18.795 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecd2c4700 1 -- 172.21.15.117:0/903822614 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc009950 con 0x7feec809a110 2020-11-20T13:56:18.795 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feecd2c4700 1 -- 172.21.15.117:0/903822614 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feebc009cb0 con 0x7feec809a110 2020-11-20T13:56:18.795 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 -- 172.21.15.117:0/903822614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec809a110 msgr2=0x7feec81437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.796 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.787+0000 7feed052a700 1 --2- 172.21.15.117:0/903822614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec809a110 0x7feec81437a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7feebc002f30 tx=0x7feebc006e80).stop 2020-11-20T13:56:18.796 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.788+0000 7feed052a700 1 -- 172.21.15.117:0/903822614 shutdown_connections 2020-11-20T13:56:18.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.788+0000 7feed052a700 1 --2- 172.21.15.117:0/903822614 >> v2:172.21.15.85:3300/0 conn(0x7feec8143c90 0x7feec80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.788+0000 7feed052a700 1 --2- 172.21.15.117:0/903822614 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec809cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.788+0000 7feed052a700 1 --2- 172.21.15.117:0/903822614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec809a110 0x7feec81437a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.797 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.788+0000 7feed052a700 1 -- 172.21.15.117:0/903822614 >> 172.21.15.117:0/903822614 conn(0x7feec80b9770 msgr2=0x7feec80b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.788+0000 7feed052a700 1 -- 172.21.15.117:0/903822614 shutdown_connections 2020-11-20T13:56:18.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.788+0000 7feed052a700 1 -- 172.21.15.117:0/903822614 wait complete. 2020-11-20T13:56:18.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.789+0000 7feed052a700 1 Processor -- start 2020-11-20T13:56:18.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.789+0000 7feed052a700 1 -- start start 2020-11-20T13:56:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feec809a110 0x7feec809dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec80b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec8143c90 0x7feec813ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec813c3b0 con 0x7feec809a110 2020-11-20T13:56:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec8156f50 con 0x7feec8143c90 2020-11-20T13:56:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec813c5a0 con 0x7feec809bd80 2020-11-20T13:56:18.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec8143c90 0x7feec813ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec8143c90 0x7feec813ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41270/0 (socket says 172.21.15.117:41270) 2020-11-20T13:56:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 -- 172.21.15.117:0/3096788941 learned_addr learned my addr 172.21.15.117:0/3096788941 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feecdac5700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec80b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feece2c6700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feec809a110 0x7feec809dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 -- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 msgr2=0x7feec80b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec80b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 -- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feec809a110 msgr2=0x7feec809dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feec809a110 0x7feec809dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 -- 172.21.15.117:0/3096788941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8156c80 con 0x7feec8143c90 2020-11-20T13:56:18.804 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feeceac7700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec8143c90 0x7feec813ca40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7feec4006910 tx=0x7feec4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.805 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feebb7fe700 1 -- 172.21.15.117:0/3096788941 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feec40127e0 con 0x7feec8143c90 2020-11-20T13:56:18.805 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feebb7fe700 1 -- 172.21.15.117:0/3096788941 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4012940 con 0x7feec8143c90 2020-11-20T13:56:18.805 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec809e390 con 0x7feec8143c90 2020-11-20T13:56:18.805 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feebb7fe700 1 -- 172.21.15.117:0/3096788941 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feec4012ca0 con 0x7feec8143c90 2020-11-20T13:56:18.805 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.790+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec809f020 con 0x7feec8143c90 2020-11-20T13:56:18.806 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.791+0000 7feebb7fe700 1 -- 172.21.15.117:0/3096788941 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7feec4013480 con 0x7feec8143c90 2020-11-20T13:56:18.807 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.791+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec8057050 con 0x7feec8143c90 2020-11-20T13:56:18.807 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.791+0000 7feebb7fe700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feea80513c0 0x7feea8053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:18.807 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.792+0000 7feebb7fe700 1 -- 172.21.15.117:0/3096788941 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7feec4069740 con 0x7feec8143c90 2020-11-20T13:56:18.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.792+0000 7feece2c6700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feea80513c0 0x7feea8053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:18.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.792+0000 7feece2c6700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feea80513c0 0x7feea8053880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7feec000d000 tx=0x7feec0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:18.808 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.795+0000 7feebb7fe700 1 -- 172.21.15.117:0/3096788941 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7feec4041900 con 0x7feec8143c90 2020-11-20T13:56:18.944 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.942+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feec814d2f0 con 0x7feec8143c90 2020-11-20T13:56:18.944 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.943+0000 7feebb7fe700 1 -- 172.21.15.117:0/3096788941 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+10 (secure 0 0 0) 0x7feec4044f20 con 0x7feec8143c90 2020-11-20T13:56:18.945 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:18.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feea80513c0 msgr2=0x7feea8053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feea80513c0 0x7feea8053880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7feec000d000 tx=0x7feec0008040).stop 2020-11-20T13:56:18.946 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec8143c90 msgr2=0x7feec813ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:18.947 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec8143c90 0x7feec813ca40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7feec4006910 tx=0x7feec4008040).stop 2020-11-20T13:56:18.947 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 shutdown_connections 2020-11-20T13:56:18.947 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feea80513c0 0x7feea8053880 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.947 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feec8143c90 0x7feec813ca40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.947 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feec809a110 0x7feec809dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 --2- 172.21.15.117:0/3096788941 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feec809bd80 0x7feec80b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:18.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 >> 172.21.15.117:0/3096788941 conn(0x7feec80b9770 msgr2=0x7feec80b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:18.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 shutdown_connections 2020-11-20T13:56:18.948 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:18.944+0000 7feed052a700 1 -- 172.21.15.117:0/3096788941 wait complete. 2020-11-20T13:56:18.958 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:19.066 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.065+0000 7f74b48fd700 1 Processor -- start 2020-11-20T13:56:19.067 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.065+0000 7f74b48fd700 1 -- start start 2020-11-20T13:56:19.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74b48fd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b0144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74b48fd700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f74b0149420 0x7f74b0145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74b48fd700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b0148a40 con 0x7f74b009b930 2020-11-20T13:56:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74b48fd700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b01462f0 con 0x7f74b0149420 2020-11-20T13:56:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74b48fd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0148ce0 con 0x7f74b0098dc0 2020-11-20T13:56:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74aed9d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f74b0149420 0x7f74b0145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74ae59c700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f74b009b930 0x7f74b01447b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:44338/0 (socket says 172.21.15.117:44338) 2020-11-20T13:56:19.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74ae59c700 1 -- 172.21.15.117:0/3200489552 learned_addr learned my addr 172.21.15.117:0/3200489552 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:19.071 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74a7fff700 1 --2- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b0144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.071 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74aed9d700 1 -- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 msgr2=0x7f74b0144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.071 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74aed9d700 1 --2- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b0144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.071 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74aed9d700 1 -- 172.21.15.117:0/3200489552 >> v1:172.21.15.85:6789/0 conn(0x7f74b009b930 legacy=0x7f74b01447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.071 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74aed9d700 1 -- 172.21.15.117:0/3200489552 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b0148f80 con 0x7f74b0149420 2020-11-20T13:56:19.072 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.066+0000 7f74aed9d700 1 --2- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0145e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f74a80032b0 tx=0x7f74a80091a0).ready entity=mon.0 client_cookie=f7decb20285c2949 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.072 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74add9b700 1 -- 172.21.15.117:0/3200489552 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a800b9a0 con 0x7f74b0149420 2020-11-20T13:56:19.072 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74add9b700 1 -- 172.21.15.117:0/3200489552 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a800bb00 con 0x7f74b0149420 2020-11-20T13:56:19.072 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74add9b700 1 -- 172.21.15.117:0/3200489552 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74a800bc70 con 0x7f74b0149420 2020-11-20T13:56:19.072 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 -- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 msgr2=0x7f74b0145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 --2- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0145e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f74a80032b0 tx=0x7f74a80091a0).stop 2020-11-20T13:56:19.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 -- 172.21.15.117:0/3200489552 shutdown_connections 2020-11-20T13:56:19.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 --2- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0145e00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 --2- 172.21.15.117:0/3200489552 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b0144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 -- 172.21.15.117:0/3200489552 >> 172.21.15.117:0/3200489552 conn(0x7f74b00b9770 msgr2=0x7f74b00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.074 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 -- 172.21.15.117:0/3200489552 shutdown_connections 2020-11-20T13:56:19.074 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.067+0000 7f74b48fd700 1 -- 172.21.15.117:0/3200489552 wait complete. 2020-11-20T13:56:19.074 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.068+0000 7f74b48fd700 1 Processor -- start 2020-11-20T13:56:19.074 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.068+0000 7f74b48fd700 1 -- start start 2020-11-20T13:56:19.077 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74b48fd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b00674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.077 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74b48fd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b009b930 0x7f74b0143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.077 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74b48fd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74b48fd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b00b37b0 con 0x7f74b0149420 2020-11-20T13:56:19.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74b48fd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b00b3a10 con 0x7f74b009b930 2020-11-20T13:56:19.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74b48fd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b00b3cb0 con 0x7f74b0098dc0 2020-11-20T13:56:19.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b009b930 0x7f74b0143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.078 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b009b930 0x7f74b0143210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41280/0 (socket says 172.21.15.117:41280) 2020-11-20T13:56:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 -- 172.21.15.117:0/1833911495 learned_addr learned my addr 172.21.15.117:0/1833911495 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74aed9d700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74ae59c700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b00674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 -- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 msgr2=0x7f74b00674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.079 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b00674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 -- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 msgr2=0x7f74b0142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 -- 172.21.15.117:0/1833911495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b00b2940 con 0x7f74b009b930 2020-11-20T13:56:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74aed9d700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a7fff700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b009b930 0x7f74b0143210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f749800c000 tx=0x7f7498006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.080 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a77fe700 1 -- 172.21.15.117:0/1833911495 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f749800ca10 con 0x7f74b009b930 2020-11-20T13:56:19.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a77fe700 1 -- 172.21.15.117:0/1833911495 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749800cb70 con 0x7f74b009b930 2020-11-20T13:56:19.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b00b2b90 con 0x7f74b009b930 2020-11-20T13:56:19.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.069+0000 7f74a77fe700 1 -- 172.21.15.117:0/1833911495 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f749800ee80 con 0x7f74b009b930 2020-11-20T13:56:19.081 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.070+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b00b30b0 con 0x7f74b009b930 2020-11-20T13:56:19.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.071+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b0143700 con 0x7f74b009b930 2020-11-20T13:56:19.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.071+0000 7f74a77fe700 1 -- 172.21.15.117:0/1833911495 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f749801c460 con 0x7f74b009b930 2020-11-20T13:56:19.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.071+0000 7f74a77fe700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7494051320 0x7f74940537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.071+0000 7f74a77fe700 1 -- 172.21.15.117:0/1833911495 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f74980695e0 con 0x7f74b009b930 2020-11-20T13:56:19.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.071+0000 7f74ae59c700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7494051320 0x7f74940537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.082 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.071+0000 7f74ae59c700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7494051320 0x7f74940537e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f74a000d000 tx=0x7f74a0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.083 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.074+0000 7f74a77fe700 1 -- 172.21.15.117:0/1833911495 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f74980417a0 con 0x7f74b009b930 2020-11-20T13:56:19.227 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.225+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74b0057050 con 0x7f74b009b930 2020-11-20T13:56:19.227 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.226+0000 7f74a77fe700 1 -- 172.21.15.117:0/1833911495 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7498044dc0 con 0x7f74b009b930 2020-11-20T13:56:19.228 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:19.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7494051320 msgr2=0x7f74940537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7494051320 0x7f74940537e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f74a000d000 tx=0x7f74a0009040).stop 2020-11-20T13:56:19.229 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b009b930 msgr2=0x7f74b0143210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b009b930 0x7f74b0143210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f749800c000 tx=0x7f7498006dc0).stop 2020-11-20T13:56:19.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 shutdown_connections 2020-11-20T13:56:19.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7494051320 0x7f74940537e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74b0098dc0 0x7f74b00674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.230 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74b0149420 0x7f74b0142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.227+0000 7f74b48fd700 1 --2- 172.21.15.117:0/1833911495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b009b930 0x7f74b0143210 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.228+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 >> 172.21.15.117:0/1833911495 conn(0x7f74b00b9770 msgr2=0x7f74b009bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.228+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 shutdown_connections 2020-11-20T13:56:19.231 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.228+0000 7f74b48fd700 1 -- 172.21.15.117:0/1833911495 wait complete. 2020-11-20T13:56:19.247 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) 2020-11-20T13:56:19.248 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early log 'Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair' 2020-11-20T13:56:19.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.355+0000 7f637e6eb700 1 Processor -- start 2020-11-20T13:56:19.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.355+0000 7f637e6eb700 1 -- start start 2020-11-20T13:56:19.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f637e6eb700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 0x7f63780aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f637e6eb700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f6378099fa0 0x7f637809a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f637e6eb700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63780b5120 con 0x7f63780b5860 2020-11-20T13:56:19.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f637e6eb700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63780b4f10 con 0x7f6378099fa0 2020-11-20T13:56:19.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f637e6eb700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63780b53c0 con 0x7f63780ae8b0 2020-11-20T13:56:19.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f63777fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f63780b5860 0x7f6378098b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60446/0 (socket says 172.21.15.117:60446) 2020-11-20T13:56:19.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f63777fe700 1 -- 172.21.15.117:0/1798554403 learned_addr learned my addr 172.21.15.117:0/1798554403 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:19.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f637cc88700 1 --2- 172.21.15.117:0/1798554403 >> v2:172.21.15.85:3300/0 conn(0x7f6378099fa0 0x7f637809a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f6377fff700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 0x7f63780aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731734320 0 0) 0x7f63780b5120 con 0x7f63780b5860 2020-11-20T13:56:19.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.356+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6364003230 con 0x7f63780b5860 2020-11-20T13:56:19.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2203342576 0 0) 0x7f6364003230 con 0x7f63780b5860 2020-11-20T13:56:19.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 msgr2=0x7f63780aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 0x7f63780aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 >> v2:172.21.15.85:3300/0 conn(0x7f6378099fa0 msgr2=0x7f637809a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 --2- 172.21.15.117:0/1798554403 >> v2:172.21.15.85:3300/0 conn(0x7f6378099fa0 0x7f637809a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63780b4ac0 con 0x7f63780b5860 2020-11-20T13:56:19.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f636c0016d0 con 0x7f63780b5860 2020-11-20T13:56:19.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 >> v1:172.21.15.117:6789/0 conn(0x7f63780b5860 legacy=0x7f6378098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6364003720 0x7f6364003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 0x7f63640044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f63640049c0 0x7f6364004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780b4f10 con 0x7f63640049c0 2020-11-20T13:56:19.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780b53c0 con 0x7f6364003720 2020-11-20T13:56:19.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63640041c0 con 0x7f63780ae8b0 2020-11-20T13:56:19.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6377fff700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6364003720 0x7f6364003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f63777fe700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 0x7f63640044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f637cc88700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f63640049c0 0x7f6364004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6377fff700 1 -- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 msgr2=0x7f63640044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6377fff700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f63780ae8b0 0x7f63640044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6377fff700 1 -- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f63640049c0 msgr2=0x7f6364004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:19.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6377fff700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f63640049c0 0x7f6364004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6377fff700 1 -- 172.21.15.117:0/1798554403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6364003230 con 0x7f6364003720 2020-11-20T13:56:19.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f637cc88700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f63640049c0 0x7f6364004e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:19.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6377fff700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6364003720 0x7f6364003b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f636800d000 tx=0x7f6368006e50).ready entity=mon.1 client_cookie=c6b40e781376f2ff server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f636800fb90 con 0x7f6364003720 2020-11-20T13:56:19.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636800fcf0 con 0x7f6364003720 2020-11-20T13:56:19.370 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f6376ffd700 1 -- 172.21.15.117:0/1798554403 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6368012650 con 0x7f6364003720 2020-11-20T13:56:19.370 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f637e6eb700 1 -- 172.21.15.117:0/1798554403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6364003720 msgr2=0x7f6364003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.370 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f637e6eb700 1 --2- 172.21.15.117:0/1798554403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6364003720 0x7f6364003b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f636800d000 tx=0x7f6368006e50).stop 2020-11-20T13:56:19.370 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.357+0000 7f637cc88700 1 -- 172.21.15.117:0/1798554403 reap_dead start 2020-11-20T13:56:19.370 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.358+0000 7f637e6eb700 1 -- 172.21.15.117:0/1798554403 shutdown_connections 2020-11-20T13:56:19.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.358+0000 7f637e6eb700 1 -- 172.21.15.117:0/1798554403 >> 172.21.15.117:0/1798554403 conn(0x7f637805b9f0 msgr2=0x7f637805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:19.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.358+0000 7f637e6eb700 1 -- 172.21.15.117:0/1798554403 shutdown_connections 2020-11-20T13:56:19.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.358+0000 7f637e6eb700 1 -- 172.21.15.117:0/1798554403 wait complete. 2020-11-20T13:56:19.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.359+0000 7f637e6eb700 1 Processor -- start 2020-11-20T13:56:19.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.359+0000 7f637e6eb700 1 -- start start 2020-11-20T13:56:19.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63780b29d0 0x7f6378154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f637808f980 0x7f6378065380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6378066c90 0x7f63780670f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63781550a0 con 0x7f637808f980 2020-11-20T13:56:19.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780649a0 con 0x7f63780b29d0 2020-11-20T13:56:19.376 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6378064c40 con 0x7f6378066c90 2020-11-20T13:56:19.377 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63780b29d0 0x7f6378154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.377 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63780b29d0 0x7f6378154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41296/0 (socket says 172.21.15.117:41296) 2020-11-20T13:56:19.377 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 -- 172.21.15.117:0/1257644008 learned_addr learned my addr 172.21.15.117:0/1257644008 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:19.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637cc88700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6378066c90 0x7f63780670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f63777fe700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f637808f980 0x7f6378065380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 -- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6378066c90 msgr2=0x7f63780670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6378066c90 0x7f63780670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 -- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f637808f980 msgr2=0x7f6378065380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:19.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f637808f980 0x7f6378065380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:19.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 -- 172.21.15.117:0/1257644008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6378064ee0 con 0x7f63780b29d0 2020-11-20T13:56:19.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f6377fff700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63780b29d0 0x7f6378154bb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f636800d000 tx=0x7f6368006e50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f63757fa700 1 -- 172.21.15.117:0/1257644008 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6368006bd0 con 0x7f63780b29d0 2020-11-20T13:56:19.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f63757fa700 1 -- 172.21.15.117:0/1257644008 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636801b660 con 0x7f63780b29d0 2020-11-20T13:56:19.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63780b2db0 con 0x7f63780b29d0 2020-11-20T13:56:19.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f63757fa700 1 -- 172.21.15.117:0/1257644008 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f636801b9c0 con 0x7f63780b29d0 2020-11-20T13:56:19.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.360+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63780675e0 con 0x7f63780b29d0 2020-11-20T13:56:19.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.361+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6378065070 con 0x7f63780b29d0 2020-11-20T13:56:19.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.362+0000 7f63757fa700 1 -- 172.21.15.117:0/1257644008 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f6368029020 con 0x7f63780b29d0 2020-11-20T13:56:19.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.362+0000 7f63757fa700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6358051350 0x7f6358053810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:19.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.362+0000 7f63757fa700 1 -- 172.21.15.117:0/1257644008 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f636806ad20 con 0x7f63780b29d0 2020-11-20T13:56:19.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.362+0000 7f63777fe700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6358051350 0x7f6358053810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:19.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.362+0000 7f63777fe700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6358051350 0x7f6358053810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f636c005000 tx=0x7f636c0077c0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:19.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.365+0000 7f63757fa700 1 -- 172.21.15.117:0/1257644008 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6368046970 con 0x7f63780b29d0 2020-11-20T13:56:19.570 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:19.568+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]} v 0) v1 -- 0x7f6378057050 con 0x7f63780b29d0 2020-11-20T13:56:20.417 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.413+0000 7f63757fa700 1 -- 172.21.15.117:0/1257644008 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]}]=0 v51) v1 ==== 151+0+0 (secure 0 0 0) 0x7f6368046560 con 0x7f63780b29d0 2020-11-20T13:56:20.417 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6358051350 msgr2=0x7f6358053810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.418 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6358051350 0x7f6358053810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f636c005000 tx=0x7f636c0077c0).stop 2020-11-20T13:56:20.418 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63780b29d0 msgr2=0x7f6378154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.418 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63780b29d0 0x7f6378154bb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f636800d000 tx=0x7f6368006e50).stop 2020-11-20T13:56:20.418 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 shutdown_connections 2020-11-20T13:56:20.418 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6358051350 0x7f6358053810 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.419 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6378066c90 0x7f63780670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.419 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63780b29d0 0x7f6378154bb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.419 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 --2- 172.21.15.117:0/1257644008 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f637808f980 0x7f6378065380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.419 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 >> 172.21.15.117:0/1257644008 conn(0x7f637805b9f0 msgr2=0x7f637809e890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:20.419 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 shutdown_connections 2020-11-20T13:56:20.420 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.415+0000 7f637e6eb700 1 -- 172.21.15.117:0/1257644008 wait complete. 2020-11-20T13:56:20.433 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mon mon_allow_pool_delete True 2020-11-20T13:56:20.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 Processor -- start 2020-11-20T13:56:20.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 -- start start 2020-11-20T13:56:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062910 0x7f181809aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f1818062cf0 0x7f181805f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f181809af30 0x7f181805fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181809b3c0 con 0x7f1818062cf0 2020-11-20T13:56:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181809b660 con 0x7f181809af30 2020-11-20T13:56:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181e8f6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181809b900 con 0x7f1818062910 2020-11-20T13:56:20.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.547+0000 7f181ce93700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f181809af30 0x7f181805fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f181809af30 0x7f181805fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41308/0 (socket says 172.21.15.117:41308) 2020-11-20T13:56:20.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 -- 172.21.15.117:0/669642304 learned_addr learned my addr 172.21.15.117:0/669642304 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:20.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f180ffff700 1 --2- 172.21.15.117:0/669642304 >> v2:172.21.15.85:3300/0 conn(0x7f1818062cf0 0x7f181805f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f1817fff700 1 --2- 172.21.15.117:0/669642304 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062910 0x7f181809aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 -- 172.21.15.117:0/669642304 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062910 msgr2=0x7f181809aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:20.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 --2- 172.21.15.117:0/669642304 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062910 0x7f181809aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 -- 172.21.15.117:0/669642304 >> v2:172.21.15.85:3300/0 conn(0x7f1818062cf0 msgr2=0x7f181805f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:20.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 --2- 172.21.15.117:0/669642304 >> v2:172.21.15.85:3300/0 conn(0x7f1818062cf0 0x7f181805f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 -- 172.21.15.117:0/669642304 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18180b4310 con 0x7f181809af30 2020-11-20T13:56:20.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181ce93700 1 --2- 172.21.15.117:0/669642304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f181809af30 0x7f181805fe00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f18100032b0 tx=0x7f1810009270).ready entity=mon.1 client_cookie=b41426b9dda725ac server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f18177fe700 1 -- 172.21.15.117:0/669642304 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f181000bc00 con 0x7f181809af30 2020-11-20T13:56:20.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f18177fe700 1 -- 172.21.15.117:0/669642304 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181000bd60 con 0x7f181809af30 2020-11-20T13:56:20.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f18177fe700 1 -- 172.21.15.117:0/669642304 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1810011400 con 0x7f181809af30 2020-11-20T13:56:20.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181e8f6700 1 -- 172.21.15.117:0/669642304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f181809af30 msgr2=0x7f181805fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.548+0000 7f181e8f6700 1 --2- 172.21.15.117:0/669642304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f181809af30 0x7f181805fe00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f18100032b0 tx=0x7f1810009270).stop 2020-11-20T13:56:20.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.549+0000 7f181e8f6700 1 -- 172.21.15.117:0/669642304 shutdown_connections 2020-11-20T13:56:20.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.549+0000 7f181e8f6700 1 --2- 172.21.15.117:0/669642304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f181809af30 0x7f181805fe00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.549+0000 7f181e8f6700 1 --2- 172.21.15.117:0/669642304 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062910 0x7f181809aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.549+0000 7f181e8f6700 1 --2- 172.21.15.117:0/669642304 >> v2:172.21.15.85:3300/0 conn(0x7f1818062cf0 0x7f181805f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.549+0000 7f181e8f6700 1 -- 172.21.15.117:0/669642304 >> 172.21.15.117:0/669642304 conn(0x7f18180b9770 msgr2=0x7f18180b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:20.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.549+0000 7f181e8f6700 1 -- 172.21.15.117:0/669642304 shutdown_connections 2020-11-20T13:56:20.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.549+0000 7f181e8f6700 1 -- 172.21.15.117:0/669642304 wait complete. 2020-11-20T13:56:20.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 Processor -- start 2020-11-20T13:56:20.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 -- start start 2020-11-20T13:56:20.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1818062910 0x7f1818148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062cf0 0x7f1818148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18180b0d50 0x7f181815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b1500 con 0x7f18180b0d50 2020-11-20T13:56:20.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b1260 con 0x7f1818062910 2020-11-20T13:56:20.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181e8f6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1818066260 con 0x7f1818062cf0 2020-11-20T13:56:20.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f1817fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1818062910 0x7f1818148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f1817fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1818062910 0x7f1818148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41310/0 (socket says 172.21.15.117:41310) 2020-11-20T13:56:20.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f1817fff700 1 -- 172.21.15.117:0/624638314 learned_addr learned my addr 172.21.15.117:0/624638314 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:20.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f180ffff700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062cf0 0x7f1818148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.550+0000 7f181ce93700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18180b0d50 0x7f181815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1817fff700 1 -- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062cf0 msgr2=0x7f1818148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1817fff700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062cf0 0x7f1818148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1817fff700 1 -- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18180b0d50 msgr2=0x7f181815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1817fff700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18180b0d50 0x7f181815dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1817fff700 1 -- 172.21.15.117:0/624638314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1818067a40 con 0x7f1818062910 2020-11-20T13:56:20.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1817fff700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1818062910 0x7f1818148420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f1808003070 tx=0x7f1808008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1815ffb700 1 -- 172.21.15.117:0/624638314 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1808009790 con 0x7f1818062910 2020-11-20T13:56:20.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1815ffb700 1 -- 172.21.15.117:0/624638314 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18080098f0 con 0x7f1818062910 2020-11-20T13:56:20.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1818063b50 con 0x7f1818062910 2020-11-20T13:56:20.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f1815ffb700 1 -- 172.21.15.117:0/624638314 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1808009c50 con 0x7f1818062910 2020-11-20T13:56:20.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.551+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181815e550 con 0x7f1818062910 2020-11-20T13:56:20.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.552+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1818057050 con 0x7f1818062910 2020-11-20T13:56:20.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.552+0000 7f1815ffb700 1 -- 172.21.15.117:0/624638314 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1808013480 con 0x7f1818062910 2020-11-20T13:56:20.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.552+0000 7f1815ffb700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f17fc0513c0 0x7f17fc053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:20.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.552+0000 7f1815ffb700 1 -- 172.21.15.117:0/624638314 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f18080697f0 con 0x7f1818062910 2020-11-20T13:56:20.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.552+0000 7f180ffff700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f17fc0513c0 0x7f17fc053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:20.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.553+0000 7f180ffff700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f17fc0513c0 0x7f17fc053880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f180000c000 tx=0x7f1800008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:20.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.556+0000 7f1815ffb700 1 -- 172.21.15.117:0/624638314 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f18080419b0 con 0x7f1818062910 2020-11-20T13:56:20.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.704+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f181809f770 con 0x7f1818062910 2020-11-20T13:56:20.710 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.708+0000 7f1815ffb700 1 -- 172.21.15.117:0/624638314 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1808009db0 con 0x7f1818062910 2020-11-20T13:56:20.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f17fc0513c0 msgr2=0x7f17fc053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f17fc0513c0 0x7f17fc053880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f180000c000 tx=0x7f1800008040).stop 2020-11-20T13:56:20.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1818062910 msgr2=0x7f1818148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:20.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1818062910 0x7f1818148420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f1808003070 tx=0x7f1808008040).stop 2020-11-20T13:56:20.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 shutdown_connections 2020-11-20T13:56:20.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f17fc0513c0 0x7f17fc053880 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18180b0d50 0x7f181815dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1818062910 0x7f1818148420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 --2- 172.21.15.117:0/624638314 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1818062cf0 0x7f1818148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:20.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 >> 172.21.15.117:0/624638314 conn(0x7f18180b9770 msgr2=0x7f18180b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:20.713 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 shutdown_connections 2020-11-20T13:56:20.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:20.710+0000 7f181e8f6700 1 -- 172.21.15.117:0/624638314 wait complete. 2020-11-20T13:56:20.721 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T13:56:20.722 INFO:teuthology.orchestra.run.smithi085:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T13:56:20.796 INFO:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T13:56:20.865 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T13:56:20.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:20.866 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:20.866 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:20.889 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:20.889 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:20.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:20.890 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:20.891 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:20.944 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:20.945 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:20.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:20.946 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:20.946 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2020-11-20T13:56:20.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:56:21.000 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:21.000 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi117.front.sepia.ceph.com... 2020-11-20T13:56:21.000 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T13:56:21.001 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:21.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.078 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.078 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:21.132 INFO:teuthology.orchestra.run.smithi117.stdout:ext2/ext3 2020-11-20T13:56:21.133 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-20T13:56:21.133 INFO:teuthology.orchestra.run:waiting for 900 2020-11-20T13:56:21.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T13:56:21.134 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:21.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.186 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.187 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:21.242 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:21.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-20T13:56:21.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.243 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.244 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:21.298 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:21.299 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2020-11-20T13:56:21.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.300 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.300 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:21.354 INFO:teuthology.orchestra.run.smithi117.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2020-11-20T13:56:21.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:56:21.355 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2020-11-20T13:56:21.355 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi117.front.sepia.ceph.com... 2020-11-20T13:56:21.356 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T13:56:21.356 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:21.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.433 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.433 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:21.486 INFO:teuthology.orchestra.run.smithi117.stdout:ext2/ext3 2020-11-20T13:56:21.487 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-20T13:56:21.487 INFO:teuthology.orchestra.run:waiting for 900 2020-11-20T13:56:21.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T13:56:21.488 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T13:56:21.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.540 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.541 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:21.560 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2020-11-20T13:56:21.560 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-11-20T13:56:21.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.561 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.561 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:21.617 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2020-11-20T13:56:21.618 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2020-11-20T13:56:21.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.619 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.619 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:21.675 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': Transport endpoint is not connected 2020-11-20T13:56:21.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:56:21.676 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.2 2020-11-20T13:56:21.677 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2020-11-20T13:56:21.678 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T13:56:21.678 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:21.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.759 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.759 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:21.815 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2020-11-20T13:56:21.816 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-20T13:56:21.817 INFO:teuthology.orchestra.run:waiting for 900 2020-11-20T13:56:21.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T13:56:21.818 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T13:56:21.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.871 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.871 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:21.927 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2020-11-20T13:56:21.928 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2020-11-20T13:56:21.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.929 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.930 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:21.984 INFO:teuthology.orchestra.run.smithi085.stdout:fuseblk 2020-11-20T13:56:21.984 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2020-11-20T13:56:21.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:21.985 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:21.988 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:22.019 INFO:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:22.026 INFO:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:22.040 INFO:teuthology.orchestra.run.smithi085.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.3': Transport endpoint is not connected 2020-11-20T13:56:22.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T13:56:22.041 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.3 2020-11-20T13:56:22.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi085.front.sepia.ceph.com... 2020-11-20T13:56:22.042 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T13:56:22.042 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:22.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:22.123 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:22.123 INFO:teuthology.orchestra.run.smithi085:> ' -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:22.179 INFO:teuthology.orchestra.run.smithi085.stdout:ext2/ext3 2020-11-20T13:56:22.180 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-20T13:56:22.180 INFO:teuthology.orchestra.run:waiting for 900 2020-11-20T13:56:22.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T13:56:22.181 INFO:teuthology.orchestra.run.smithi085:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T13:56:22.237 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json-pretty 2020-11-20T13:56:22.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.346+0000 7f57859d8700 1 Processor -- start 2020-11-20T13:56:22.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.346+0000 7f57859d8700 1 -- start start 2020-11-20T13:56:22.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.346+0000 7f57859d8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5780098e30 0x7f578009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.346+0000 7f57859d8700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f57800ae6f0 0x7f57800aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f57859d8700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57800b5510 con 0x7f578009b700 2020-11-20T13:56:22.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f57859d8700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57800b57b0 con 0x7f57800ae6f0 2020-11-20T13:56:22.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f57859d8700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57800b5a50 con 0x7f5780098e30 2020-11-20T13:56:22.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577e7fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f578009b700 0x7f57801438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60478/0 (socket says 172.21.15.117:60478) 2020-11-20T13:56:22.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577e7fc700 1 -- 172.21.15.117:0/605512491 learned_addr learned my addr 172.21.15.117:0/605512491 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:22.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577effd700 1 --2- 172.21.15.117:0/605512491 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5780098e30 0x7f578009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577f7fe700 1 --2- 172.21.15.117:0/605512491 >> v2:172.21.15.85:3300/0 conn(0x7f57800ae6f0 0x7f57800aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577dffb700 1 -- 172.21.15.117:0/605512491 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595748193 0 0) 0x7f57800b5510 con 0x7f578009b700 2020-11-20T13:56:22.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577dffb700 1 -- 172.21.15.117:0/605512491 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5760003230 con 0x7f578009b700 2020-11-20T13:56:22.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577effd700 1 -- 172.21.15.117:0/605512491 >> v2:172.21.15.85:3300/0 conn(0x7f57800ae6f0 msgr2=0x7f57800aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:22.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577effd700 1 --2- 172.21.15.117:0/605512491 >> v2:172.21.15.85:3300/0 conn(0x7f57800ae6f0 0x7f57800aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577effd700 1 -- 172.21.15.117:0/605512491 >> v1:172.21.15.117:6789/0 conn(0x7f578009b700 legacy=0x7f57801438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577effd700 1 -- 172.21.15.117:0/605512491 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780099210 con 0x7f5780098e30 2020-11-20T13:56:22.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577f7fe700 1 --2- 172.21.15.117:0/605512491 >> v2:172.21.15.85:3300/0 conn(0x7f57800ae6f0 0x7f57800aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:22.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577effd700 1 --2- 172.21.15.117:0/605512491 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5780098e30 0x7f578009b210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f57780032b0 tx=0x7f57780091a0).ready entity=mon.2 client_cookie=56d8e1a196283cdd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577dffb700 1 -- 172.21.15.117:0/605512491 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f577800b9a0 con 0x7f5780098e30 2020-11-20T13:56:22.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577dffb700 1 -- 172.21.15.117:0/605512491 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577800bb00 con 0x7f5780098e30 2020-11-20T13:56:22.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f577dffb700 1 -- 172.21.15.117:0/605512491 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f577800bc70 con 0x7f5780098e30 2020-11-20T13:56:22.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f57859d8700 1 -- 172.21.15.117:0/605512491 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5780098e30 msgr2=0x7f578009b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.347+0000 7f57859d8700 1 --2- 172.21.15.117:0/605512491 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5780098e30 0x7f578009b210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f57780032b0 tx=0x7f57780091a0).stop 2020-11-20T13:56:22.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.348+0000 7f57859d8700 1 -- 172.21.15.117:0/605512491 shutdown_connections 2020-11-20T13:56:22.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.348+0000 7f57859d8700 1 --2- 172.21.15.117:0/605512491 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5780098e30 0x7f578009b210 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.348+0000 7f57859d8700 1 --2- 172.21.15.117:0/605512491 >> v2:172.21.15.85:3300/0 conn(0x7f57800ae6f0 0x7f57800aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.348+0000 7f57859d8700 1 -- 172.21.15.117:0/605512491 >> 172.21.15.117:0/605512491 conn(0x7f578005b9f0 msgr2=0x7f578005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:22.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.348+0000 7f57859d8700 1 -- 172.21.15.117:0/605512491 shutdown_connections 2020-11-20T13:56:22.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.348+0000 7f57859d8700 1 -- 172.21.15.117:0/605512491 wait complete. 2020-11-20T13:56:22.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.349+0000 7f57859d8700 1 Processor -- start 2020-11-20T13:56:22.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.349+0000 7f57859d8700 1 -- start start 2020-11-20T13:56:22.362 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5780098e30 0x7f578009c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f578009b700 0x7f5780066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f57800ae6f0 0x7f57801438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801442d0 con 0x7f578009b700 2020-11-20T13:56:22.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801444e0 con 0x7f5780098e30 2020-11-20T13:56:22.363 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5780144780 con 0x7f57800ae6f0 2020-11-20T13:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5780098e30 0x7f578009c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5780098e30 0x7f578009c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41324/0 (socket says 172.21.15.117:41324) 2020-11-20T13:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 -- 172.21.15.117:0/2741687325 learned_addr learned my addr 172.21.15.117:0/2741687325 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577e7fc700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f578009b700 0x7f5780066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577f7fe700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f57800ae6f0 0x7f57801438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 -- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f57800ae6f0 msgr2=0x7f57801438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f57800ae6f0 0x7f57801438c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 -- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f578009b700 msgr2=0x7f5780066f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f578009b700 0x7f5780066f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 -- 172.21.15.117:0/2741687325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578009e4b0 con 0x7f5780098e30 2020-11-20T13:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f577effd700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5780098e30 0x7f578009c7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f57780032b0 tx=0x7f5778016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f5777fff700 1 -- 172.21.15.117:0/2741687325 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f577801f3f0 con 0x7f5780098e30 2020-11-20T13:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f5777fff700 1 -- 172.21.15.117:0/2741687325 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577801f550 con 0x7f5780098e30 2020-11-20T13:56:22.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5780143db0 con 0x7f5780098e30 2020-11-20T13:56:22.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f5777fff700 1 -- 172.21.15.117:0/2741687325 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f577801f8b0 con 0x7f5780098e30 2020-11-20T13:56:22.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.350+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578009d380 con 0x7f5780098e30 2020-11-20T13:56:22.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.351+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5780144060 con 0x7f5780098e30 2020-11-20T13:56:22.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.351+0000 7f5777fff700 1 -- 172.21.15.117:0/2741687325 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5778026030 con 0x7f5780098e30 2020-11-20T13:56:22.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.351+0000 7f5777fff700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5764051210 0x7f57640536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.368 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.352+0000 7f5777fff700 1 -- 172.21.15.117:0/2741687325 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f5778069c50 con 0x7f5780098e30 2020-11-20T13:56:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.352+0000 7f577e7fc700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5764051210 0x7f57640536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.352+0000 7f577e7fc700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5764051210 0x7f57640536d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f576800d000 tx=0x7f5768009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.355+0000 7f5777fff700 1 -- 172.21.15.117:0/2741687325 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5778041e70 con 0x7f5780098e30 2020-11-20T13:56:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.505+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5780057050 con 0x7f5780098e30 2020-11-20T13:56:22.507 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.505+0000 7f5777fff700 1 -- 172.21.15.117:0/2741687325 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+34113 (secure 0 0 0) 0x7f5778024090 con 0x7f5780098e30 2020-11-20T13:56:22.507 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:22.508 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2020-11-20T13:56:22.508 INFO:teuthology.orchestra.run.smithi117.stdout: "epoch": 25, 2020-11-20T13:56:22.508 INFO:teuthology.orchestra.run.smithi117.stdout: "fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", 2020-11-20T13:56:22.509 INFO:teuthology.orchestra.run.smithi117.stdout: "created": "2020-11-20T13:55:30.198818+0000", 2020-11-20T13:56:22.509 INFO:teuthology.orchestra.run.smithi117.stdout: "modified": "2020-11-20T13:55:51.676406+0000", 2020-11-20T13:56:22.509 INFO:teuthology.orchestra.run.smithi117.stdout: "last_up_change": "2020-11-20T13:55:39.144730+0000", 2020-11-20T13:56:22.509 INFO:teuthology.orchestra.run.smithi117.stdout: "last_in_change": "2020-11-20T13:55:39.144730+0000", 2020-11-20T13:56:22.509 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-20T13:56:22.510 INFO:teuthology.orchestra.run.smithi117.stdout: "flags_num": 5799936, 2020-11-20T13:56:22.510 INFO:teuthology.orchestra.run.smithi117.stdout: "flags_set": [ 2020-11-20T13:56:22.510 INFO:teuthology.orchestra.run.smithi117.stdout: "pglog_hardlimit", 2020-11-20T13:56:22.510 INFO:teuthology.orchestra.run.smithi117.stdout: "purged_snapdirs", 2020-11-20T13:56:22.511 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_deletes", 2020-11-20T13:56:22.511 INFO:teuthology.orchestra.run.smithi117.stdout: "sortbitwise" 2020-11-20T13:56:22.511 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2020-11-20T13:56:22.511 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_version": 5, 2020-11-20T13:56:22.511 INFO:teuthology.orchestra.run.smithi117.stdout: "full_ratio": 0.94999998807907104, 2020-11-20T13:56:22.512 INFO:teuthology.orchestra.run.smithi117.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-20T13:56:22.512 INFO:teuthology.orchestra.run.smithi117.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-20T13:56:22.512 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_snapshot": "", 2020-11-20T13:56:22.512 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_max": 4, 2020-11-20T13:56:22.512 INFO:teuthology.orchestra.run.smithi117.stdout: "max_osd": 8, 2020-11-20T13:56:22.513 INFO:teuthology.orchestra.run.smithi117.stdout: "require_min_compat_client": "jewel", 2020-11-20T13:56:22.513 INFO:teuthology.orchestra.run.smithi117.stdout: "min_compat_client": "jewel", 2020-11-20T13:56:22.513 INFO:teuthology.orchestra.run.smithi117.stdout: "require_osd_release": "octopus", 2020-11-20T13:56:22.513 INFO:teuthology.orchestra.run.smithi117.stdout: "pools": [ 2020-11-20T13:56:22.514 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.514 INFO:teuthology.orchestra.run.smithi117.stdout: "pool": 1, 2020-11-20T13:56:22.514 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": "device_health_metrics", 2020-11-20T13:56:22.514 INFO:teuthology.orchestra.run.smithi117.stdout: "create_time": "2020-11-20T13:55:34.161117+0000", 2020-11-20T13:56:22.514 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2020-11-20T13:56:22.515 INFO:teuthology.orchestra.run.smithi117.stdout: "flags_names": "hashpspool", 2020-11-20T13:56:22.515 INFO:teuthology.orchestra.run.smithi117.stdout: "type": 1, 2020-11-20T13:56:22.515 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 2, 2020-11-20T13:56:22.515 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": 1, 2020-11-20T13:56:22.515 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_rule": 0, 2020-11-20T13:56:22.516 INFO:teuthology.orchestra.run.smithi117.stdout: "object_hash": 2, 2020-11-20T13:56:22.516 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscale_mode": "off", 2020-11-20T13:56:22.516 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": 1, 2020-11-20T13:56:22.516 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num": 1, 2020-11-20T13:56:22.516 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num_target": 1, 2020-11-20T13:56:22.517 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_target": 1, 2020-11-20T13:56:22.517 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_pending": 1, 2020-11-20T13:56:22.517 INFO:teuthology.orchestra.run.smithi117.stdout: "last_pg_merge_meta": { 2020-11-20T13:56:22.517 INFO:teuthology.orchestra.run.smithi117.stdout: "source_pgid": "0.0", 2020-11-20T13:56:22.518 INFO:teuthology.orchestra.run.smithi117.stdout: "ready_epoch": 0, 2020-11-20T13:56:22.518 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_started": 0, 2020-11-20T13:56:22.518 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_clean": 0, 2020-11-20T13:56:22.518 INFO:teuthology.orchestra.run.smithi117.stdout: "source_version": "0'0", 2020-11-20T13:56:22.518 INFO:teuthology.orchestra.run.smithi117.stdout: "target_version": "0'0" 2020-11-20T13:56:22.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.519 INFO:teuthology.orchestra.run.smithi117.stdout: "last_change": "15", 2020-11-20T13:56:22.519 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend": "0", 2020-11-20T13:56:22.519 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T13:56:22.519 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T13:56:22.520 INFO:teuthology.orchestra.run.smithi117.stdout: "auid": 0, 2020-11-20T13:56:22.520 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_mode": "selfmanaged", 2020-11-20T13:56:22.520 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_seq": 0, 2020-11-20T13:56:22.520 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_epoch": 0, 2020-11-20T13:56:22.520 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_snaps": [], 2020-11-20T13:56:22.521 INFO:teuthology.orchestra.run.smithi117.stdout: "removed_snaps": "[]", 2020-11-20T13:56:22.521 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_bytes": 0, 2020-11-20T13:56:22.521 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_objects": 0, 2020-11-20T13:56:22.521 INFO:teuthology.orchestra.run.smithi117.stdout: "tiers": [], 2020-11-20T13:56:22.522 INFO:teuthology.orchestra.run.smithi117.stdout: "tier_of": -1, 2020-11-20T13:56:22.522 INFO:teuthology.orchestra.run.smithi117.stdout: "read_tier": -1, 2020-11-20T13:56:22.522 INFO:teuthology.orchestra.run.smithi117.stdout: "write_tier": -1, 2020-11-20T13:56:22.522 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_mode": "none", 2020-11-20T13:56:22.522 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_bytes": 0, 2020-11-20T13:56:22.523 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_objects": 0, 2020-11-20T13:56:22.523 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T13:56:22.523 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T13:56:22.523 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T13:56:22.523 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_flush_age": 0, 2020-11-20T13:56:22.524 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_evict_age": 0, 2020-11-20T13:56:22.524 INFO:teuthology.orchestra.run.smithi117.stdout: "erasure_code_profile": "", 2020-11-20T13:56:22.524 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_params": { 2020-11-20T13:56:22.524 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "none" 2020-11-20T13:56:22.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.525 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_period": 0, 2020-11-20T13:56:22.525 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_count": 0, 2020-11-20T13:56:22.525 INFO:teuthology.orchestra.run.smithi117.stdout: "use_gmt_hitset": true, 2020-11-20T13:56:22.525 INFO:teuthology.orchestra.run.smithi117.stdout: "min_read_recency_for_promote": 0, 2020-11-20T13:56:22.526 INFO:teuthology.orchestra.run.smithi117.stdout: "min_write_recency_for_promote": 0, 2020-11-20T13:56:22.526 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T13:56:22.526 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_search_last_n": 0, 2020-11-20T13:56:22.526 INFO:teuthology.orchestra.run.smithi117.stdout: "grade_table": [], 2020-11-20T13:56:22.526 INFO:teuthology.orchestra.run.smithi117.stdout: "stripe_width": 0, 2020-11-20T13:56:22.527 INFO:teuthology.orchestra.run.smithi117.stdout: "expected_num_objects": 0, 2020-11-20T13:56:22.527 INFO:teuthology.orchestra.run.smithi117.stdout: "fast_read": false, 2020-11-20T13:56:22.527 INFO:teuthology.orchestra.run.smithi117.stdout: "options": { 2020-11-20T13:56:22.527 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_min": 1 2020-11-20T13:56:22.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.528 INFO:teuthology.orchestra.run.smithi117.stdout: "application_metadata": { 2020-11-20T13:56:22.528 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr_devicehealth": {} 2020-11-20T13:56:22.528 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.528 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.529 INFO:teuthology.orchestra.run.smithi117.stdout: "pool": 2, 2020-11-20T13:56:22.529 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": "rbd", 2020-11-20T13:56:22.529 INFO:teuthology.orchestra.run.smithi117.stdout: "create_time": "2020-11-20T13:55:42.995788+0000", 2020-11-20T13:56:22.530 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2020-11-20T13:56:22.530 INFO:teuthology.orchestra.run.smithi117.stdout: "flags_names": "hashpspool", 2020-11-20T13:56:22.530 INFO:teuthology.orchestra.run.smithi117.stdout: "type": 1, 2020-11-20T13:56:22.530 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 2, 2020-11-20T13:56:22.530 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": 1, 2020-11-20T13:56:22.531 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_rule": 0, 2020-11-20T13:56:22.531 INFO:teuthology.orchestra.run.smithi117.stdout: "object_hash": 2, 2020-11-20T13:56:22.531 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscale_mode": "off", 2020-11-20T13:56:22.531 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": 8, 2020-11-20T13:56:22.531 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num": 8, 2020-11-20T13:56:22.532 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num_target": 8, 2020-11-20T13:56:22.532 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_target": 8, 2020-11-20T13:56:22.532 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_pending": 8, 2020-11-20T13:56:22.532 INFO:teuthology.orchestra.run.smithi117.stdout: "last_pg_merge_meta": { 2020-11-20T13:56:22.532 INFO:teuthology.orchestra.run.smithi117.stdout: "source_pgid": "0.0", 2020-11-20T13:56:22.533 INFO:teuthology.orchestra.run.smithi117.stdout: "ready_epoch": 0, 2020-11-20T13:56:22.533 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_started": 0, 2020-11-20T13:56:22.533 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_clean": 0, 2020-11-20T13:56:22.533 INFO:teuthology.orchestra.run.smithi117.stdout: "source_version": "0'0", 2020-11-20T13:56:22.534 INFO:teuthology.orchestra.run.smithi117.stdout: "target_version": "0'0" 2020-11-20T13:56:22.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.535 INFO:teuthology.orchestra.run.smithi117.stdout: "last_change": "18", 2020-11-20T13:56:22.535 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend": "0", 2020-11-20T13:56:22.536 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T13:56:22.536 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T13:56:22.536 INFO:teuthology.orchestra.run.smithi117.stdout: "auid": 0, 2020-11-20T13:56:22.536 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_mode": "selfmanaged", 2020-11-20T13:56:22.536 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_seq": 0, 2020-11-20T13:56:22.537 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_epoch": 0, 2020-11-20T13:56:22.537 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_snaps": [], 2020-11-20T13:56:22.537 INFO:teuthology.orchestra.run.smithi117.stdout: "removed_snaps": "[]", 2020-11-20T13:56:22.537 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_bytes": 0, 2020-11-20T13:56:22.538 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_objects": 0, 2020-11-20T13:56:22.538 INFO:teuthology.orchestra.run.smithi117.stdout: "tiers": [], 2020-11-20T13:56:22.538 INFO:teuthology.orchestra.run.smithi117.stdout: "tier_of": -1, 2020-11-20T13:56:22.539 INFO:teuthology.orchestra.run.smithi117.stdout: "read_tier": -1, 2020-11-20T13:56:22.539 INFO:teuthology.orchestra.run.smithi117.stdout: "write_tier": -1, 2020-11-20T13:56:22.539 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_mode": "none", 2020-11-20T13:56:22.539 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_bytes": 0, 2020-11-20T13:56:22.539 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_objects": 0, 2020-11-20T13:56:22.540 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T13:56:22.540 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T13:56:22.540 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T13:56:22.540 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_flush_age": 0, 2020-11-20T13:56:22.540 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_evict_age": 0, 2020-11-20T13:56:22.541 INFO:teuthology.orchestra.run.smithi117.stdout: "erasure_code_profile": "", 2020-11-20T13:56:22.541 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_params": { 2020-11-20T13:56:22.541 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "none" 2020-11-20T13:56:22.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.542 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_period": 0, 2020-11-20T13:56:22.542 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_count": 0, 2020-11-20T13:56:22.542 INFO:teuthology.orchestra.run.smithi117.stdout: "use_gmt_hitset": true, 2020-11-20T13:56:22.542 INFO:teuthology.orchestra.run.smithi117.stdout: "min_read_recency_for_promote": 0, 2020-11-20T13:56:22.542 INFO:teuthology.orchestra.run.smithi117.stdout: "min_write_recency_for_promote": 0, 2020-11-20T13:56:22.543 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T13:56:22.543 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_search_last_n": 0, 2020-11-20T13:56:22.543 INFO:teuthology.orchestra.run.smithi117.stdout: "grade_table": [], 2020-11-20T13:56:22.543 INFO:teuthology.orchestra.run.smithi117.stdout: "stripe_width": 0, 2020-11-20T13:56:22.543 INFO:teuthology.orchestra.run.smithi117.stdout: "expected_num_objects": 0, 2020-11-20T13:56:22.544 INFO:teuthology.orchestra.run.smithi117.stdout: "fast_read": false, 2020-11-20T13:56:22.544 INFO:teuthology.orchestra.run.smithi117.stdout: "options": {}, 2020-11-20T13:56:22.544 INFO:teuthology.orchestra.run.smithi117.stdout: "application_metadata": { 2020-11-20T13:56:22.544 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd": {} 2020-11-20T13:56:22.544 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.545 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.545 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.545 INFO:teuthology.orchestra.run.smithi117.stdout: "pool": 3, 2020-11-20T13:56:22.545 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": "cephfs_metadata", 2020-11-20T13:56:22.546 INFO:teuthology.orchestra.run.smithi117.stdout: "create_time": "2020-11-20T13:55:45.628162+0000", 2020-11-20T13:56:22.546 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2020-11-20T13:56:22.546 INFO:teuthology.orchestra.run.smithi117.stdout: "flags_names": "hashpspool", 2020-11-20T13:56:22.546 INFO:teuthology.orchestra.run.smithi117.stdout: "type": 1, 2020-11-20T13:56:22.546 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 2, 2020-11-20T13:56:22.547 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": 1, 2020-11-20T13:56:22.547 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_rule": 0, 2020-11-20T13:56:22.547 INFO:teuthology.orchestra.run.smithi117.stdout: "object_hash": 2, 2020-11-20T13:56:22.547 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscale_mode": "off", 2020-11-20T13:56:22.549 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": 8, 2020-11-20T13:56:22.549 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num": 8, 2020-11-20T13:56:22.549 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num_target": 8, 2020-11-20T13:56:22.549 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_target": 8, 2020-11-20T13:56:22.550 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_pending": 8, 2020-11-20T13:56:22.550 INFO:teuthology.orchestra.run.smithi117.stdout: "last_pg_merge_meta": { 2020-11-20T13:56:22.550 INFO:teuthology.orchestra.run.smithi117.stdout: "source_pgid": "0.0", 2020-11-20T13:56:22.550 INFO:teuthology.orchestra.run.smithi117.stdout: "ready_epoch": 0, 2020-11-20T13:56:22.551 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_started": 0, 2020-11-20T13:56:22.551 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_clean": 0, 2020-11-20T13:56:22.551 INFO:teuthology.orchestra.run.smithi117.stdout: "source_version": "0'0", 2020-11-20T13:56:22.551 INFO:teuthology.orchestra.run.smithi117.stdout: "target_version": "0'0" 2020-11-20T13:56:22.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.552 INFO:teuthology.orchestra.run.smithi117.stdout: "last_change": "24", 2020-11-20T13:56:22.552 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend": "0", 2020-11-20T13:56:22.552 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T13:56:22.552 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T13:56:22.553 INFO:teuthology.orchestra.run.smithi117.stdout: "auid": 0, 2020-11-20T13:56:22.553 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_mode": "selfmanaged", 2020-11-20T13:56:22.553 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_seq": 0, 2020-11-20T13:56:22.553 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_epoch": 0, 2020-11-20T13:56:22.553 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_snaps": [], 2020-11-20T13:56:22.554 INFO:teuthology.orchestra.run.smithi117.stdout: "removed_snaps": "[]", 2020-11-20T13:56:22.554 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_bytes": 0, 2020-11-20T13:56:22.554 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_objects": 0, 2020-11-20T13:56:22.554 INFO:teuthology.orchestra.run.smithi117.stdout: "tiers": [], 2020-11-20T13:56:22.554 INFO:teuthology.orchestra.run.smithi117.stdout: "tier_of": -1, 2020-11-20T13:56:22.555 INFO:teuthology.orchestra.run.smithi117.stdout: "read_tier": -1, 2020-11-20T13:56:22.555 INFO:teuthology.orchestra.run.smithi117.stdout: "write_tier": -1, 2020-11-20T13:56:22.555 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_mode": "none", 2020-11-20T13:56:22.555 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_bytes": 0, 2020-11-20T13:56:22.555 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_objects": 0, 2020-11-20T13:56:22.556 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T13:56:22.556 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T13:56:22.556 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T13:56:22.556 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_flush_age": 0, 2020-11-20T13:56:22.556 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_evict_age": 0, 2020-11-20T13:56:22.557 INFO:teuthology.orchestra.run.smithi117.stdout: "erasure_code_profile": "", 2020-11-20T13:56:22.557 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_params": { 2020-11-20T13:56:22.557 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "none" 2020-11-20T13:56:22.557 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.557 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_period": 0, 2020-11-20T13:56:22.558 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_count": 0, 2020-11-20T13:56:22.558 INFO:teuthology.orchestra.run.smithi117.stdout: "use_gmt_hitset": true, 2020-11-20T13:56:22.558 INFO:teuthology.orchestra.run.smithi117.stdout: "min_read_recency_for_promote": 0, 2020-11-20T13:56:22.558 INFO:teuthology.orchestra.run.smithi117.stdout: "min_write_recency_for_promote": 0, 2020-11-20T13:56:22.559 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T13:56:22.559 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_search_last_n": 0, 2020-11-20T13:56:22.559 INFO:teuthology.orchestra.run.smithi117.stdout: "grade_table": [], 2020-11-20T13:56:22.560 INFO:teuthology.orchestra.run.smithi117.stdout: "stripe_width": 0, 2020-11-20T13:56:22.560 INFO:teuthology.orchestra.run.smithi117.stdout: "expected_num_objects": 0, 2020-11-20T13:56:22.560 INFO:teuthology.orchestra.run.smithi117.stdout: "fast_read": false, 2020-11-20T13:56:22.560 INFO:teuthology.orchestra.run.smithi117.stdout: "options": { 2020-11-20T13:56:22.561 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscale_bias": 4, 2020-11-20T13:56:22.561 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_min": 16, 2020-11-20T13:56:22.561 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_priority": 5 2020-11-20T13:56:22.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.562 INFO:teuthology.orchestra.run.smithi117.stdout: "application_metadata": { 2020-11-20T13:56:22.562 INFO:teuthology.orchestra.run.smithi117.stdout: "cephfs": { 2020-11-20T13:56:22.562 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": "cephfs" 2020-11-20T13:56:22.562 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.562 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.563 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.563 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.563 INFO:teuthology.orchestra.run.smithi117.stdout: "pool": 4, 2020-11-20T13:56:22.563 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": "cephfs_data", 2020-11-20T13:56:22.563 INFO:teuthology.orchestra.run.smithi117.stdout: "create_time": "2020-11-20T13:55:48.335298+0000", 2020-11-20T13:56:22.564 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 5, 2020-11-20T13:56:22.564 INFO:teuthology.orchestra.run.smithi117.stdout: "flags_names": "hashpspool,ec_overwrites", 2020-11-20T13:56:22.564 INFO:teuthology.orchestra.run.smithi117.stdout: "type": 3, 2020-11-20T13:56:22.564 INFO:teuthology.orchestra.run.smithi117.stdout: "size": 4, 2020-11-20T13:56:22.564 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": 3, 2020-11-20T13:56:22.565 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_rule": 1, 2020-11-20T13:56:22.565 INFO:teuthology.orchestra.run.smithi117.stdout: "object_hash": 2, 2020-11-20T13:56:22.565 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscale_mode": "off", 2020-11-20T13:56:22.565 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": 8, 2020-11-20T13:56:22.565 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num": 8, 2020-11-20T13:56:22.566 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_placement_num_target": 8, 2020-11-20T13:56:22.566 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_target": 8, 2020-11-20T13:56:22.566 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num_pending": 8, 2020-11-20T13:56:22.566 INFO:teuthology.orchestra.run.smithi117.stdout: "last_pg_merge_meta": { 2020-11-20T13:56:22.567 INFO:teuthology.orchestra.run.smithi117.stdout: "source_pgid": "0.0", 2020-11-20T13:56:22.567 INFO:teuthology.orchestra.run.smithi117.stdout: "ready_epoch": 0, 2020-11-20T13:56:22.567 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_started": 0, 2020-11-20T13:56:22.567 INFO:teuthology.orchestra.run.smithi117.stdout: "last_epoch_clean": 0, 2020-11-20T13:56:22.568 INFO:teuthology.orchestra.run.smithi117.stdout: "source_version": "0'0", 2020-11-20T13:56:22.568 INFO:teuthology.orchestra.run.smithi117.stdout: "target_version": "0'0" 2020-11-20T13:56:22.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.568 INFO:teuthology.orchestra.run.smithi117.stdout: "last_change": "25", 2020-11-20T13:56:22.568 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend": "0", 2020-11-20T13:56:22.569 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T13:56:22.569 INFO:teuthology.orchestra.run.smithi117.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T13:56:22.569 INFO:teuthology.orchestra.run.smithi117.stdout: "auid": 0, 2020-11-20T13:56:22.569 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_mode": "selfmanaged", 2020-11-20T13:56:22.569 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_seq": 0, 2020-11-20T13:56:22.570 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_epoch": 0, 2020-11-20T13:56:22.570 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_snaps": [], 2020-11-20T13:56:22.570 INFO:teuthology.orchestra.run.smithi117.stdout: "removed_snaps": "[]", 2020-11-20T13:56:22.570 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_bytes": 0, 2020-11-20T13:56:22.571 INFO:teuthology.orchestra.run.smithi117.stdout: "quota_max_objects": 0, 2020-11-20T13:56:22.571 INFO:teuthology.orchestra.run.smithi117.stdout: "tiers": [], 2020-11-20T13:56:22.571 INFO:teuthology.orchestra.run.smithi117.stdout: "tier_of": -1, 2020-11-20T13:56:22.571 INFO:teuthology.orchestra.run.smithi117.stdout: "read_tier": -1, 2020-11-20T13:56:22.571 INFO:teuthology.orchestra.run.smithi117.stdout: "write_tier": -1, 2020-11-20T13:56:22.572 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_mode": "none", 2020-11-20T13:56:22.572 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_bytes": 0, 2020-11-20T13:56:22.572 INFO:teuthology.orchestra.run.smithi117.stdout: "target_max_objects": 0, 2020-11-20T13:56:22.572 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T13:56:22.573 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T13:56:22.573 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T13:56:22.573 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_flush_age": 0, 2020-11-20T13:56:22.573 INFO:teuthology.orchestra.run.smithi117.stdout: "cache_min_evict_age": 0, 2020-11-20T13:56:22.573 INFO:teuthology.orchestra.run.smithi117.stdout: "erasure_code_profile": "cephfs_data", 2020-11-20T13:56:22.574 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_params": { 2020-11-20T13:56:22.574 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "none" 2020-11-20T13:56:22.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.574 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_period": 0, 2020-11-20T13:56:22.574 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_count": 0, 2020-11-20T13:56:22.575 INFO:teuthology.orchestra.run.smithi117.stdout: "use_gmt_hitset": true, 2020-11-20T13:56:22.575 INFO:teuthology.orchestra.run.smithi117.stdout: "min_read_recency_for_promote": 0, 2020-11-20T13:56:22.575 INFO:teuthology.orchestra.run.smithi117.stdout: "min_write_recency_for_promote": 0, 2020-11-20T13:56:22.575 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T13:56:22.575 INFO:teuthology.orchestra.run.smithi117.stdout: "hit_set_search_last_n": 0, 2020-11-20T13:56:22.576 INFO:teuthology.orchestra.run.smithi117.stdout: "grade_table": [], 2020-11-20T13:56:22.576 INFO:teuthology.orchestra.run.smithi117.stdout: "stripe_width": 8192, 2020-11-20T13:56:22.576 INFO:teuthology.orchestra.run.smithi117.stdout: "expected_num_objects": 0, 2020-11-20T13:56:22.576 INFO:teuthology.orchestra.run.smithi117.stdout: "fast_read": false, 2020-11-20T13:56:22.576 INFO:teuthology.orchestra.run.smithi117.stdout: "options": {}, 2020-11-20T13:56:22.577 INFO:teuthology.orchestra.run.smithi117.stdout: "application_metadata": { 2020-11-20T13:56:22.577 INFO:teuthology.orchestra.run.smithi117.stdout: "cephfs": { 2020-11-20T13:56:22.577 INFO:teuthology.orchestra.run.smithi117.stdout: "data": "cephfs" 2020-11-20T13:56:22.577 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.577 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.578 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.578 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2020-11-20T13:56:22.578 INFO:teuthology.orchestra.run.smithi117.stdout: "osds": [ 2020-11-20T13:56:22.578 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.578 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 0, 2020-11-20T13:56:22.579 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "6425c8e9-c695-4b85-8225-09d3cf8763e7", 2020-11-20T13:56:22.579 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.579 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.579 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.579 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.580 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.580 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.580 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.580 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 22, 2020-11-20T13:56:22.580 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.581 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.581 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.581 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.581 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.581 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.582 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6802", 2020-11-20T13:56:22.582 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.582 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.582 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.583 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6803", 2020-11-20T13:56:22.583 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.583 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.583 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.584 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.584 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.584 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.584 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.584 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6804", 2020-11-20T13:56:22.585 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.586 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.586 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.586 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6805", 2020-11-20T13:56:22.586 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.587 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.587 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.588 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.588 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.588 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.588 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.589 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6808", 2020-11-20T13:56:22.589 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.589 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.590 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.590 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6809", 2020-11-20T13:56:22.590 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.590 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.590 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.591 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.591 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.591 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.592 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.592 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6806", 2020-11-20T13:56:22.592 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.592 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.593 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.593 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6807", 2020-11-20T13:56:22.593 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22905 2020-11-20T13:56:22.593 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.593 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.594 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.117:6803/22905", 2020-11-20T13:56:22.594 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.117:6805/22905", 2020-11-20T13:56:22.594 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.117:6809/22905", 2020-11-20T13:56:22.594 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.117:6807/22905", 2020-11-20T13:56:22.595 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.595 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.599 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.600 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.600 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.600 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 1, 2020-11-20T13:56:22.600 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb", 2020-11-20T13:56:22.601 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.601 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.601 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.601 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.602 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.602 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.602 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.602 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 22, 2020-11-20T13:56:22.602 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.603 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.603 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.603 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.603 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.603 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.604 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6818", 2020-11-20T13:56:22.604 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.604 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.604 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.605 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6819", 2020-11-20T13:56:22.605 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.605 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.606 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.606 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.606 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.607 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.607 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.607 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6820", 2020-11-20T13:56:22.608 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.608 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.608 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.608 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6821", 2020-11-20T13:56:22.609 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.609 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.609 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.613 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.614 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.614 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.614 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.614 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6824", 2020-11-20T13:56:22.614 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.615 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.615 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.615 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6825", 2020-11-20T13:56:22.616 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.616 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.616 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.616 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.616 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.617 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.621 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.621 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.621 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6822", 2020-11-20T13:56:22.621 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.622 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.622 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.622 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6823", 2020-11-20T13:56:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22907 2020-11-20T13:56:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.623 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.117:6819/22907", 2020-11-20T13:56:22.624 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.117:6821/22907", 2020-11-20T13:56:22.624 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.117:6825/22907", 2020-11-20T13:56:22.624 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.117:6823/22907", 2020-11-20T13:56:22.624 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.624 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.625 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.625 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.625 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.625 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 2, 2020-11-20T13:56:22.626 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "2aaa1ba8-9363-4263-b86b-b163296df2dc", 2020-11-20T13:56:22.626 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.626 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.626 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.626 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.627 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.627 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.627 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.627 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 16, 2020-11-20T13:56:22.629 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.629 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.629 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.629 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.630 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.630 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.630 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6810", 2020-11-20T13:56:22.630 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.631 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.631 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.631 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6811", 2020-11-20T13:56:22.631 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.631 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.632 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.632 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.632 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.632 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.633 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.633 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6812", 2020-11-20T13:56:22.633 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.634 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.634 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.634 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6813", 2020-11-20T13:56:22.634 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.634 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.635 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.635 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.635 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.636 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.636 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.636 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6816", 2020-11-20T13:56:22.640 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.640 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.641 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.641 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6817", 2020-11-20T13:56:22.641 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.641 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.641 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.642 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.642 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.642 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.642 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.643 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6814", 2020-11-20T13:56:22.643 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.643 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.644 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.644 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6815", 2020-11-20T13:56:22.644 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22906 2020-11-20T13:56:22.644 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.644 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.645 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.117:6811/22906", 2020-11-20T13:56:22.645 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.117:6813/22906", 2020-11-20T13:56:22.645 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.117:6817/22906", 2020-11-20T13:56:22.645 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.117:6815/22906", 2020-11-20T13:56:22.646 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.646 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.646 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.646 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.647 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.647 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 3, 2020-11-20T13:56:22.647 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "61687ded-23a5-4350-aa98-f5be6d396d11", 2020-11-20T13:56:22.648 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.648 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.648 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.649 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.649 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.649 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.649 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.650 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 22, 2020-11-20T13:56:22.650 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.650 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.650 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.651 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.651 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.651 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.651 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6826", 2020-11-20T13:56:22.651 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.652 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.652 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.652 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6827", 2020-11-20T13:56:22.653 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.653 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.653 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.653 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.654 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.654 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.654 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.654 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6828", 2020-11-20T13:56:22.654 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.655 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.655 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.655 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6829", 2020-11-20T13:56:22.656 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.656 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.656 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.656 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.657 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.657 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.657 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.657 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6832", 2020-11-20T13:56:22.657 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.658 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.658 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.658 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6833", 2020-11-20T13:56:22.658 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.659 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.659 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.659 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.659 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.660 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.660 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.660 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6830", 2020-11-20T13:56:22.660 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.661 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.661 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.661 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6831", 2020-11-20T13:56:22.661 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 22922 2020-11-20T13:56:22.662 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.662 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.662 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.117:6827/22922", 2020-11-20T13:56:22.662 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.117:6829/22922", 2020-11-20T13:56:22.663 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.117:6833/22922", 2020-11-20T13:56:22.663 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.117:6831/22922", 2020-11-20T13:56:22.663 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.663 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.663 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.664 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.664 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.664 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 4, 2020-11-20T13:56:22.664 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "48d54979-1a08-4759-846e-b56eb855d218", 2020-11-20T13:56:22.665 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.665 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.665 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.665 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.665 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.666 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.666 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.666 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 22, 2020-11-20T13:56:22.666 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.667 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.667 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.667 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.667 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.667 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.668 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6800", 2020-11-20T13:56:22.668 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.668 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.668 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.669 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6801", 2020-11-20T13:56:22.669 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.669 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.669 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.670 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.670 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.670 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.670 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.670 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6802", 2020-11-20T13:56:22.671 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.671 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.671 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.671 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6803", 2020-11-20T13:56:22.672 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.672 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.672 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.672 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.673 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.673 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.673 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.673 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6806", 2020-11-20T13:56:22.674 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.674 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.674 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6807", 2020-11-20T13:56:22.675 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.675 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.675 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.675 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.676 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.676 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.676 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.676 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6804", 2020-11-20T13:56:22.676 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.677 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.677 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.677 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6805", 2020-11-20T13:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23126 2020-11-20T13:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.678 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.85:6801/23126", 2020-11-20T13:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.85:6803/23126", 2020-11-20T13:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.85:6807/23126", 2020-11-20T13:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.85:6805/23126", 2020-11-20T13:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.679 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.680 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 5, 2020-11-20T13:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "8e801f28-c346-40d6-a3f8-5e4d072ed72d", 2020-11-20T13:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.681 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.682 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.682 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.682 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.682 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 19, 2020-11-20T13:56:22.683 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.683 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.683 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.683 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.683 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.684 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.684 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6824", 2020-11-20T13:56:22.684 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.684 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.684 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.685 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6825", 2020-11-20T13:56:22.685 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.685 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.685 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.686 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.686 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.686 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.686 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.686 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6826", 2020-11-20T13:56:22.687 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.687 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.687 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.687 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6827", 2020-11-20T13:56:22.688 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.688 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.688 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.688 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.689 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.689 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.689 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.689 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6830", 2020-11-20T13:56:22.689 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.690 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.690 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.690 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6831", 2020-11-20T13:56:22.690 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.691 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.691 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.691 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.691 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.692 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.692 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.692 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6828", 2020-11-20T13:56:22.692 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.693 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.693 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.693 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6829", 2020-11-20T13:56:22.693 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23124 2020-11-20T13:56:22.694 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.694 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.694 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.85:6825/23124", 2020-11-20T13:56:22.694 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.85:6827/23124", 2020-11-20T13:56:22.695 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.85:6831/23124", 2020-11-20T13:56:22.695 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.85:6829/23124", 2020-11-20T13:56:22.695 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.695 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.695 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.696 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.696 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.696 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 6, 2020-11-20T13:56:22.696 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "94604f91-e895-43c8-8032-36a36e1191c3", 2020-11-20T13:56:22.697 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.697 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.697 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.697 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.697 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.698 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.698 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.698 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 22, 2020-11-20T13:56:22.698 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.698 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.699 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.699 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.699 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.699 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.699 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6815", 2020-11-20T13:56:22.700 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.700 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.700 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.700 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6817", 2020-11-20T13:56:22.701 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.701 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.701 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6818", 2020-11-20T13:56:22.702 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.703 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.703 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.703 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6819", 2020-11-20T13:56:22.703 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.704 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.704 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.704 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.704 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.705 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.705 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.705 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6822", 2020-11-20T13:56:22.705 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.706 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.706 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.706 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6823", 2020-11-20T13:56:22.706 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.706 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.707 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.707 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.707 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.707 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.708 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.708 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6820", 2020-11-20T13:56:22.708 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.708 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.709 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.709 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6821", 2020-11-20T13:56:22.709 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23127 2020-11-20T13:56:22.709 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.709 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.710 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.85:6817/23127", 2020-11-20T13:56:22.710 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.85:6819/23127", 2020-11-20T13:56:22.710 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.85:6823/23127", 2020-11-20T13:56:22.710 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.85:6821/23127", 2020-11-20T13:56:22.711 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.711 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.711 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.711 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.712 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 7, 2020-11-20T13:56:22.712 INFO:teuthology.orchestra.run.smithi117.stdout: "uuid": "0b70a2e6-1111-426a-a7d9-5ccdd88b4038", 2020-11-20T13:56:22.712 INFO:teuthology.orchestra.run.smithi117.stdout: "up": 1, 2020-11-20T13:56:22.712 INFO:teuthology.orchestra.run.smithi117.stdout: "in": 1, 2020-11-20T13:56:22.713 INFO:teuthology.orchestra.run.smithi117.stdout: "weight": 1, 2020-11-20T13:56:22.713 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_affinity": 1, 2020-11-20T13:56:22.713 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_begin": 0, 2020-11-20T13:56:22.713 INFO:teuthology.orchestra.run.smithi117.stdout: "last_clean_end": 0, 2020-11-20T13:56:22.713 INFO:teuthology.orchestra.run.smithi117.stdout: "up_from": 13, 2020-11-20T13:56:22.714 INFO:teuthology.orchestra.run.smithi117.stdout: "up_thru": 16, 2020-11-20T13:56:22.714 INFO:teuthology.orchestra.run.smithi117.stdout: "down_at": 0, 2020-11-20T13:56:22.714 INFO:teuthology.orchestra.run.smithi117.stdout: "lost_at": 0, 2020-11-20T13:56:22.714 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addrs": { 2020-11-20T13:56:22.714 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.715 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.715 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.715 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6808", 2020-11-20T13:56:22.715 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.716 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.716 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.716 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6809", 2020-11-20T13:56:22.716 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.716 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.717 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.717 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addrs": { 2020-11-20T13:56:22.717 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.717 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.718 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.718 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6810", 2020-11-20T13:56:22.718 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.718 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.719 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.719 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6811", 2020-11-20T13:56:22.719 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.719 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.719 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.720 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addrs": { 2020-11-20T13:56:22.720 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.720 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.720 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.721 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6814", 2020-11-20T13:56:22.721 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.721 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.721 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.722 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6816", 2020-11-20T13:56:22.722 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.722 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.723 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.723 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addrs": { 2020-11-20T13:56:22.723 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2020-11-20T13:56:22.723 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.724 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2020-11-20T13:56:22.724 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6812", 2020-11-20T13:56:22.724 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.724 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.724 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:22.725 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.85:6813", 2020-11-20T13:56:22.725 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 23128 2020-11-20T13:56:22.725 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.725 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.726 INFO:teuthology.orchestra.run.smithi117.stdout: "public_addr": "172.21.15.85:6809/23128", 2020-11-20T13:56:22.726 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_addr": "172.21.15.85:6811/23128", 2020-11-20T13:56:22.726 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_back_addr": "172.21.15.85:6816/23128", 2020-11-20T13:56:22.726 INFO:teuthology.orchestra.run.smithi117.stdout: "heartbeat_front_addr": "172.21.15.85:6813/23128", 2020-11-20T13:56:22.726 INFO:teuthology.orchestra.run.smithi117.stdout: "state": [ 2020-11-20T13:56:22.727 INFO:teuthology.orchestra.run.smithi117.stdout: "exists", 2020-11-20T13:56:22.727 INFO:teuthology.orchestra.run.smithi117.stdout: "up" 2020-11-20T13:56:22.727 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:22.727 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.727 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2020-11-20T13:56:22.728 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_xinfo": [ 2020-11-20T13:56:22.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.728 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 0, 2020-11-20T13:56:22.728 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.728 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.729 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.729 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.729 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.729 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.219715+0000", 2020-11-20T13:56:22.729 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.730 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 1, 2020-11-20T13:56:22.730 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.731 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.731 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.731 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.731 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.731 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.210126+0000", 2020-11-20T13:56:22.732 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.732 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 2, 2020-11-20T13:56:22.732 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.733 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.733 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.733 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.733 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.733 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.184385+0000", 2020-11-20T13:56:22.734 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.734 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 3, 2020-11-20T13:56:22.734 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.735 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.735 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.735 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.735 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.736 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.237566+0000", 2020-11-20T13:56:22.736 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 4, 2020-11-20T13:56:22.737 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.737 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.737 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.737 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.738 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.738 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.201769+0000", 2020-11-20T13:56:22.738 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.738 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.739 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 5, 2020-11-20T13:56:22.739 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.739 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.739 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.739 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.740 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.740 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.247931+0000", 2020-11-20T13:56:22.740 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.740 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 6, 2020-11-20T13:56:22.741 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.741 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.741 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.742 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.742 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.742 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.289808+0000", 2020-11-20T13:56:22.742 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.743 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:22.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": 7, 2020-11-20T13:56:22.743 INFO:teuthology.orchestra.run.smithi117.stdout: "down_stamp": "0.000000", 2020-11-20T13:56:22.743 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_probability": 0, 2020-11-20T13:56:22.743 INFO:teuthology.orchestra.run.smithi117.stdout: "laggy_interval": 0, 2020-11-20T13:56:22.744 INFO:teuthology.orchestra.run.smithi117.stdout: "features": 4540138292840890367, 2020-11-20T13:56:22.744 INFO:teuthology.orchestra.run.smithi117.stdout: "old_weight": 0, 2020-11-20T13:56:22.744 INFO:teuthology.orchestra.run.smithi117.stdout: "last_purged_snaps_scrub": "2020-11-20T13:55:37.198974+0000", 2020-11-20T13:56:22.744 INFO:teuthology.orchestra.run.smithi117.stdout: "dead_epoch": 0 2020-11-20T13:56:22.744 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.745 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2020-11-20T13:56:22.745 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_upmap": [], 2020-11-20T13:56:22.745 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_upmap_items": [], 2020-11-20T13:56:22.745 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_temp": [], 2020-11-20T13:56:22.745 INFO:teuthology.orchestra.run.smithi117.stdout: "primary_temp": [], 2020-11-20T13:56:22.746 INFO:teuthology.orchestra.run.smithi117.stdout: "blacklist": {}, 2020-11-20T13:56:22.746 INFO:teuthology.orchestra.run.smithi117.stdout: "erasure_code_profiles": { 2020-11-20T13:56:22.746 INFO:teuthology.orchestra.run.smithi117.stdout: "cephfs_data": { 2020-11-20T13:56:22.746 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-device-class": "", 2020-11-20T13:56:22.746 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-failure-domain": "osd", 2020-11-20T13:56:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-root": "default", 2020-11-20T13:56:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: "jerasure-per-chunk-alignment": "false", 2020-11-20T13:56:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: "k": "2", 2020-11-20T13:56:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: "m": "2", 2020-11-20T13:56:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: "plugin": "jerasure", 2020-11-20T13:56:22.748 INFO:teuthology.orchestra.run.smithi117.stdout: "ruleset-failure-domain": "osd", 2020-11-20T13:56:22.748 INFO:teuthology.orchestra.run.smithi117.stdout: "technique": "reed_sol_van", 2020-11-20T13:56:22.748 INFO:teuthology.orchestra.run.smithi117.stdout: "w": "8" 2020-11-20T13:56:22.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.748 INFO:teuthology.orchestra.run.smithi117.stdout: "default": { 2020-11-20T13:56:22.749 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-failure-domain": "osd", 2020-11-20T13:56:22.749 INFO:teuthology.orchestra.run.smithi117.stdout: "k": "2", 2020-11-20T13:56:22.749 INFO:teuthology.orchestra.run.smithi117.stdout: "m": "1", 2020-11-20T13:56:22.749 INFO:teuthology.orchestra.run.smithi117.stdout: "plugin": "jerasure", 2020-11-20T13:56:22.749 INFO:teuthology.orchestra.run.smithi117.stdout: "ruleset-failure-domain": "osd", 2020-11-20T13:56:22.750 INFO:teuthology.orchestra.run.smithi117.stdout: "technique": "reed_sol_van" 2020-11-20T13:56:22.750 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:22.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:22.750 INFO:teuthology.orchestra.run.smithi117.stdout: "removed_snaps_queue": [], 2020-11-20T13:56:22.751 INFO:teuthology.orchestra.run.smithi117.stdout: "new_removed_snaps": [], 2020-11-20T13:56:22.751 INFO:teuthology.orchestra.run.smithi117.stdout: "new_purged_snaps": [], 2020-11-20T13:56:22.751 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_node_flags": {}, 2020-11-20T13:56:22.751 INFO:teuthology.orchestra.run.smithi117.stdout: "device_class_flags": {} 2020-11-20T13:56:22.751 INFO:teuthology.orchestra.run.smithi117.stdout:} 2020-11-20T13:56:22.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5764051210 msgr2=0x7f57640536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5764051210 0x7f57640536d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f576800d000 tx=0x7f5768009040).stop 2020-11-20T13:56:22.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5780098e30 msgr2=0x7f578009c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5780098e30 0x7f578009c7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f57780032b0 tx=0x7f5778016000).stop 2020-11-20T13:56:22.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 shutdown_connections 2020-11-20T13:56:22.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5764051210 0x7f57640536d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f57800ae6f0 0x7f57801438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5780098e30 0x7f578009c7a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 --2- 172.21.15.117:0/2741687325 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f578009b700 0x7f5780066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 >> 172.21.15.117:0/2741687325 conn(0x7f578005b9f0 msgr2=0x7f57800b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 shutdown_connections 2020-11-20T13:56:22.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.507+0000 7f57859d8700 1 -- 172.21.15.117:0/2741687325 wait complete. 2020-11-20T13:56:22.756 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:22.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.862+0000 7fe7e66ad700 1 Processor -- start 2020-11-20T13:56:22.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.862+0000 7fe7e66ad700 1 -- start start 2020-11-20T13:56:22.865 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.862+0000 7fe7e66ad700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e014eff0 0x7fe7e014f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.866 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.862+0000 7fe7e66ad700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe7e0145ae0 0x7fe7e009b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.866 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.862+0000 7fe7e66ad700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e0098490 con 0x7fe7e014f8c0 2020-11-20T13:56:22.866 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.862+0000 7fe7e66ad700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e00b5100 con 0x7fe7e0145ae0 2020-11-20T13:56:22.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.862+0000 7fe7e66ad700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0098730 con 0x7fe7e014eff0 2020-11-20T13:56:22.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e4c4a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe7e0145ae0 0x7fe7e009b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7d7fff700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7fe7e014f8c0 0x7fe7e01453d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:44396/0 (socket says 172.21.15.117:44396) 2020-11-20T13:56:22.867 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7d7fff700 1 -- 172.21.15.117:0/180907653 learned_addr learned my addr 172.21.15.117:0/180907653 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:22.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e4c4a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fe7e0145ae0 0x7fe7e009b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.117:52958/0 (socket says 172.21.15.117:52958) 2020-11-20T13:56:22.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7dffff700 1 --2- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e014eff0 0x7fe7e014f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.868 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e4c4a700 1 -- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e014eff0 msgr2=0x7fe7e014f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e4c4a700 1 --2- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e014eff0 0x7fe7e014f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e4c4a700 1 -- 172.21.15.117:0/180907653 >> v1:172.21.15.85:6789/0 conn(0x7fe7e014f8c0 legacy=0x7fe7e01453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7df7fe700 1 -- 172.21.15.117:0/180907653 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646377953 0 0) 0x7fe7e0098490 con 0x7fe7e014f8c0 2020-11-20T13:56:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e4c4a700 1 -- 172.21.15.117:0/180907653 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e00989d0 con 0x7fe7e0145ae0 2020-11-20T13:56:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e4c4a700 1 --2- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e0145ae0 0x7fe7e009b6e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe7d80032b0 tx=0x7fe7d80091a0).ready entity=mon.0 client_cookie=b1698d49cbd27890 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7df7fe700 1 -- 172.21.15.117:0/180907653 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7d800b9a0 con 0x7fe7e0145ae0 2020-11-20T13:56:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7df7fe700 1 -- 172.21.15.117:0/180907653 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d800bb00 con 0x7fe7e0145ae0 2020-11-20T13:56:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7df7fe700 1 -- 172.21.15.117:0/180907653 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7d800bc70 con 0x7fe7e0145ae0 2020-11-20T13:56:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e66ad700 1 -- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e0145ae0 msgr2=0x7fe7e009b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.863+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e0145ae0 0x7fe7e009b6e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe7d80032b0 tx=0x7fe7d80091a0).stop 2020-11-20T13:56:22.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.864+0000 7fe7e66ad700 1 -- 172.21.15.117:0/180907653 shutdown_connections 2020-11-20T13:56:22.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.864+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e0145ae0 0x7fe7e009b6e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.864+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/180907653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e014eff0 0x7fe7e014f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.871 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.864+0000 7fe7e66ad700 1 -- 172.21.15.117:0/180907653 >> 172.21.15.117:0/180907653 conn(0x7fe7e005b9f0 msgr2=0x7fe7e005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:22.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.864+0000 7fe7e66ad700 1 -- 172.21.15.117:0/180907653 shutdown_connections 2020-11-20T13:56:22.872 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.864+0000 7fe7e66ad700 1 -- 172.21.15.117:0/180907653 wait complete. 2020-11-20T13:56:22.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 Processor -- start 2020-11-20T13:56:22.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 -- start start 2020-11-20T13:56:22.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e0145ae0 0x7fe7e01482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e014eff0 0x7fe7e015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e014f8c0 0x7fe7e015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0146e80 con 0x7fe7e014f8c0 2020-11-20T13:56:22.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0147120 con 0x7fe7e014eff0 2020-11-20T13:56:22.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.865+0000 7fe7e66ad700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01473c0 con 0x7fe7e0145ae0 2020-11-20T13:56:22.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e014eff0 0x7fe7e015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7dffff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e0145ae0 0x7fe7e01482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e014eff0 0x7fe7e015b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41340/0 (socket says 172.21.15.117:41340) 2020-11-20T13:56:22.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 -- 172.21.15.117:0/4029879875 learned_addr learned my addr 172.21.15.117:0/4029879875 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:22.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7e4c4a700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e014f8c0 0x7fe7e015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 -- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e0145ae0 msgr2=0x7fe7e01482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e0145ae0 0x7fe7e01482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 -- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e014f8c0 msgr2=0x7fe7e015b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:22.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e014f8c0 0x7fe7e015b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:22.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 -- 172.21.15.117:0/4029879875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e0146620 con 0x7fe7e014eff0 2020-11-20T13:56:22.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7e4c4a700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e014f8c0 0x7fe7e015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:22.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7dffff700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e0145ae0 0x7fe7e01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:22.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7d7fff700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e014eff0 0x7fe7e015b1e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe7c8000e90 tx=0x7fe7c8007a30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7ddffb700 1 -- 172.21.15.117:0/4029879875 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7c80072e0 con 0x7fe7e014eff0 2020-11-20T13:56:22.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7ddffb700 1 -- 172.21.15.117:0/4029879875 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c8007440 con 0x7fe7e014eff0 2020-11-20T13:56:22.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7ddffb700 1 -- 172.21.15.117:0/4029879875 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe7c80057d0 con 0x7fe7e014eff0 2020-11-20T13:56:22.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e0146a50 con 0x7fe7e014eff0 2020-11-20T13:56:22.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.866+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e0148990 con 0x7fe7e014eff0 2020-11-20T13:56:22.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.867+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e0147580 con 0x7fe7e014eff0 2020-11-20T13:56:22.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.867+0000 7fe7ddffb700 1 -- 172.21.15.117:0/4029879875 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe7c8010070 con 0x7fe7e014eff0 2020-11-20T13:56:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.868+0000 7fe7ddffb700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe7c4051320 0x7fe7c40537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.868+0000 7fe7ddffb700 1 -- 172.21.15.117:0/4029879875 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe7c8068700 con 0x7fe7e014eff0 2020-11-20T13:56:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.868+0000 7fe7dffff700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe7c4051320 0x7fe7c40537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.868+0000 7fe7dffff700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe7c4051320 0x7fe7c40537e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe7d000b000 tx=0x7fe7d0006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:22.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:22.872+0000 7fe7ddffb700 1 -- 172.21.15.117:0/4029879875 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe7c8040920 con 0x7fe7e014eff0 2020-11-20T13:56:23.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.070+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7e0057050 con 0x7fe7e014eff0 2020-11-20T13:56:23.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.070+0000 7fe7ddffb700 1 -- 172.21.15.117:0/4029879875 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7fe7c8043f40 con 0x7fe7e014eff0 2020-11-20T13:56:23.071 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:23.071 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:55:51.676934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/1587825839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1587825839},{"type":"v1","addr":"172.21.15.117:6835","nonce":1587825839}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe7c4051320 msgr2=0x7fe7c40537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.072 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe7c4051320 0x7fe7c40537e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe7d000b000 tx=0x7fe7d0006dc0).stop 2020-11-20T13:56:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e014eff0 msgr2=0x7fe7e015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e014eff0 0x7fe7e015b1e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe7c8000e90 tx=0x7fe7c8007a30).stop 2020-11-20T13:56:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 shutdown_connections 2020-11-20T13:56:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe7c4051320 0x7fe7c40537e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.073 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe7e0145ae0 0x7fe7e01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.074 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe7e014f8c0 0x7fe7e015b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.074 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 --2- 172.21.15.117:0/4029879875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7e014eff0 0x7fe7e015b1e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.075 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 >> 172.21.15.117:0/4029879875 conn(0x7fe7e005b9f0 msgr2=0x7fe7e0144170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:23.076 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 shutdown_connections 2020-11-20T13:56:23.076 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.071+0000 7fe7e66ad700 1 -- 172.21.15.117:0/4029879875 wait complete. 2020-11-20T13:56:23.076 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2020-11-20T13:56:23.083 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs fail cephfs 2020-11-20T13:56:23.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.190+0000 7f80ab4b9700 1 Processor -- start 2020-11-20T13:56:23.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.190+0000 7f80ab4b9700 1 -- start start 2020-11-20T13:56:23.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80ab4b9700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80a4098e30 0x7f80a409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80ab4b9700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f80a40ae6f0 0x7f80a40aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80ab4b9700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40b5510 con 0x7f80a409b700 2020-11-20T13:56:23.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80ab4b9700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40b57b0 con 0x7f80a40ae6f0 2020-11-20T13:56:23.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80ab4b9700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40b5a50 con 0x7f80a4098e30 2020-11-20T13:56:23.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80a9255700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80a4098e30 0x7f80a409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80a8a54700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f80a409b700 0x7f80a41438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60506/0 (socket says 172.21.15.117:60506) 2020-11-20T13:56:23.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80a8a54700 1 -- 172.21.15.117:0/52100748 learned_addr learned my addr 172.21.15.117:0/52100748 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:23.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80a9a56700 1 --2- 172.21.15.117:0/52100748 >> v2:172.21.15.85:3300/0 conn(0x7f80a40ae6f0 0x7f80a40aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.191+0000 7f80a9255700 1 -- 172.21.15.117:0/52100748 >> v2:172.21.15.85:3300/0 conn(0x7f80a40ae6f0 msgr2=0x7f80a40aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:23.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80a9255700 1 --2- 172.21.15.117:0/52100748 >> v2:172.21.15.85:3300/0 conn(0x7f80a40ae6f0 0x7f80a40aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80a9255700 1 -- 172.21.15.117:0/52100748 >> v1:172.21.15.117:6789/0 conn(0x7f80a409b700 legacy=0x7f80a41438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f8093fff700 1 -- 172.21.15.117:0/52100748 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726224593 0 0) 0x7f80a40b5510 con 0x7f80a409b700 2020-11-20T13:56:23.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80a9255700 1 -- 172.21.15.117:0/52100748 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4099210 con 0x7f80a4098e30 2020-11-20T13:56:23.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80a9a56700 1 --2- 172.21.15.117:0/52100748 >> v2:172.21.15.85:3300/0 conn(0x7f80a40ae6f0 0x7f80a40aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:23.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80a9255700 1 --2- 172.21.15.117:0/52100748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a409b210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f8094002f30 tx=0x7f8094006dc0).ready entity=mon.1 client_cookie=6a265e31d14c43ef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f8093fff700 1 -- 172.21.15.117:0/52100748 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80940097f0 con 0x7f80a4098e30 2020-11-20T13:56:23.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f8093fff700 1 -- 172.21.15.117:0/52100748 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8094009950 con 0x7f80a4098e30 2020-11-20T13:56:23.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f8093fff700 1 -- 172.21.15.117:0/52100748 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8094009cb0 con 0x7f80a4098e30 2020-11-20T13:56:23.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 -- 172.21.15.117:0/52100748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 msgr2=0x7f80a409b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/52100748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a409b210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f8094002f30 tx=0x7f8094006dc0).stop 2020-11-20T13:56:23.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 -- 172.21.15.117:0/52100748 shutdown_connections 2020-11-20T13:56:23.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/52100748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a409b210 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/52100748 >> v2:172.21.15.85:3300/0 conn(0x7f80a40ae6f0 0x7f80a40aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 -- 172.21.15.117:0/52100748 >> 172.21.15.117:0/52100748 conn(0x7f80a405b9f0 msgr2=0x7f80a405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:23.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 -- 172.21.15.117:0/52100748 shutdown_connections 2020-11-20T13:56:23.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.192+0000 7f80ab4b9700 1 -- 172.21.15.117:0/52100748 wait complete. 2020-11-20T13:56:23.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.193+0000 7f80ab4b9700 1 Processor -- start 2020-11-20T13:56:23.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.193+0000 7f80ab4b9700 1 -- start start 2020-11-20T13:56:23.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80ab4b9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a40b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80ab4b9700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f80a409b700 0x7f80a40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80ab4b9700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f80a40ae6f0 0x7f80a41428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80ab4b9700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41544f0 con 0x7f80a40ae6f0 2020-11-20T13:56:23.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80ab4b9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4154790 con 0x7f80a4098e30 2020-11-20T13:56:23.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80ab4b9700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4154a30 con 0x7f80a409b700 2020-11-20T13:56:23.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a40b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a40b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41350/0 (socket says 172.21.15.117:41350) 2020-11-20T13:56:23.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 -- 172.21.15.117:0/3701267339 learned_addr learned my addr 172.21.15.117:0/3701267339 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:23.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9a56700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f80a40ae6f0 0x7f80a41428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a8a54700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f80a409b700 0x7f80a40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 -- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f80a409b700 msgr2=0x7f80a40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f80a409b700 0x7f80a40b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 -- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f80a40ae6f0 msgr2=0x7f80a41428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f80a40ae6f0 0x7f80a41428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 -- 172.21.15.117:0/3701267339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a41541a0 con 0x7f80a4098e30 2020-11-20T13:56:23.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.194+0000 7f80a9255700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a40b4470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8094002f30 tx=0x7f8094015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.195+0000 7f80927fc700 1 -- 172.21.15.117:0/3701267339 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8094009e60 con 0x7f80a4098e30 2020-11-20T13:56:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.195+0000 7f80927fc700 1 -- 172.21.15.117:0/3701267339 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8094009c50 con 0x7f80a4098e30 2020-11-20T13:56:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.195+0000 7f80927fc700 1 -- 172.21.15.117:0/3701267339 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f809401d640 con 0x7f80a4098e30 2020-11-20T13:56:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.195+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a4064240 con 0x7f80a4098e30 2020-11-20T13:56:23.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.195+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a4155110 con 0x7f80a4098e30 2020-11-20T13:56:23.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.196+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4057050 con 0x7f80a4098e30 2020-11-20T13:56:23.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.196+0000 7f80927fc700 1 -- 172.21.15.117:0/3701267339 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8094024030 con 0x7f80a4098e30 2020-11-20T13:56:23.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.196+0000 7f80927fc700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8084051320 0x7f80840537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.196+0000 7f80927fc700 1 -- 172.21.15.117:0/3701267339 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6431+0+0 (secure 0 0 0) 0x7f809406a320 con 0x7f80a4098e30 2020-11-20T13:56:23.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.196+0000 7f80a8a54700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8084051320 0x7f80840537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.196+0000 7f80a8a54700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8084051320 0x7f80840537e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f80a000b000 tx=0x7f80a0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.200+0000 7f80927fc700 1 -- 172.21.15.117:0/3701267339 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8094042580 con 0x7f80a4098e30 2020-11-20T13:56:23.406 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.406+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f80a40b84b0 con 0x7f80a4098e30 2020-11-20T13:56:23.417 INFO:tasks.ceph.mon.b.smithi085.stderr:2020-11-20T13:56:23.416+0000 7f2a23f2f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T13:56:23.422 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.422+0000 7f80927fc700 1 -- 172.21.15.117:0/3701267339 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8094045b60 con 0x7f80a4098e30 2020-11-20T13:56:23.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8084051320 msgr2=0x7f80840537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8084051320 0x7f80840537e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f80a000b000 tx=0x7f80a0009040).stop 2020-11-20T13:56:23.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 msgr2=0x7f80a40b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a40b4470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8094002f30 tx=0x7f8094015000).stop 2020-11-20T13:56:23.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 shutdown_connections 2020-11-20T13:56:23.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8084051320 0x7f80840537e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f80a40ae6f0 0x7f80a41428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80a4098e30 0x7f80a40b4470 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 --2- 172.21.15.117:0/3701267339 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f80a409b700 0x7f80a40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 >> 172.21.15.117:0/3701267339 conn(0x7f80a405b9f0 msgr2=0x7f80a409c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:23.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 shutdown_connections 2020-11-20T13:56:23.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.424+0000 7f80ab4b9700 1 -- 172.21.15.117:0/3701267339 wait complete. 2020-11-20T13:56:23.427 INFO:teuthology.orchestra.run.smithi117.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T13:56:23.436 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:23.444 INFO:tasks.ceph.mds.a.smithi117.stdout:starting mds.a at 2020-11-20T13:56:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.543+0000 7f40af5d6700 1 Processor -- start 2020-11-20T13:56:23.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.543+0000 7f40af5d6700 1 -- start start 2020-11-20T13:56:23.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40af5d6700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40a81473d0 0x7f40a80b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40af5d6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a8067b40 0x7f40a80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40af5d6700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a80b4a00 con 0x7f40a80be620 2020-11-20T13:56:23.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40af5d6700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a814af20 con 0x7f40a81473d0 2020-11-20T13:56:23.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40af5d6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a814b1c0 con 0x7f40a8067b40 2020-11-20T13:56:23.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40acb71700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40a81473d0 0x7f40a80b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40acb71700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40a81473d0 0x7f40a80b8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41370/0 (socket says 172.21.15.117:41370) 2020-11-20T13:56:23.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.544+0000 7f40acb71700 1 -- 172.21.15.117:0/1998684882 learned_addr learned my addr 172.21.15.117:0/1998684882 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:23.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40adb73700 1 --2- 172.21.15.117:0/1998684882 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a8067b40 0x7f40a80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40acb71700 1 -- 172.21.15.117:0/1998684882 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a8067b40 msgr2=0x7f40a80b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:23.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40acb71700 1 --2- 172.21.15.117:0/1998684882 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a8067b40 0x7f40a80b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40acb71700 1 -- 172.21.15.117:0/1998684882 >> v1:172.21.15.85:6789/0 conn(0x7f40a80be620 legacy=0x7f40a8146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40acb71700 1 -- 172.21.15.117:0/1998684882 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a814b460 con 0x7f40a81473d0 2020-11-20T13:56:23.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40acb71700 1 --2- 172.21.15.117:0/1998684882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a81473d0 0x7f40a80b8530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f409c002f30 tx=0x7f409c006e80).ready entity=mon.1 client_cookie=9c4d98fce3ba1c01 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40a7fff700 1 -- 172.21.15.117:0/1998684882 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409c0097f0 con 0x7f40a81473d0 2020-11-20T13:56:23.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40a7fff700 1 -- 172.21.15.117:0/1998684882 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c009950 con 0x7f40a81473d0 2020-11-20T13:56:23.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40a7fff700 1 -- 172.21.15.117:0/1998684882 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409c009cb0 con 0x7f40a81473d0 2020-11-20T13:56:23.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 -- 172.21.15.117:0/1998684882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a81473d0 msgr2=0x7f40a80b8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 --2- 172.21.15.117:0/1998684882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a81473d0 0x7f40a80b8530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f409c002f30 tx=0x7f409c006e80).stop 2020-11-20T13:56:23.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 -- 172.21.15.117:0/1998684882 shutdown_connections 2020-11-20T13:56:23.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 --2- 172.21.15.117:0/1998684882 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a8067b40 0x7f40a80b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 --2- 172.21.15.117:0/1998684882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a81473d0 0x7f40a80b8530 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 -- 172.21.15.117:0/1998684882 >> 172.21.15.117:0/1998684882 conn(0x7f40a805c4c0 msgr2=0x7f40a80bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:23.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 -- 172.21.15.117:0/1998684882 shutdown_connections 2020-11-20T13:56:23.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.545+0000 7f40af5d6700 1 -- 172.21.15.117:0/1998684882 wait complete. 2020-11-20T13:56:23.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.546+0000 7f40af5d6700 1 Processor -- start 2020-11-20T13:56:23.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.546+0000 7f40af5d6700 1 -- start start 2020-11-20T13:56:23.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40af5d6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a8067b40 0x7f40a8091800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40af5d6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40a80be620 0x7f40a8149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40af5d6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a81473d0 0x7f40a815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40af5d6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8091e30 con 0x7f40a80be620 2020-11-20T13:56:23.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40af5d6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a80920d0 con 0x7f40a8067b40 2020-11-20T13:56:23.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40af5d6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8092370 con 0x7f40a81473d0 2020-11-20T13:56:23.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a8067b40 0x7f40a8091800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a8067b40 0x7f40a8091800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41374/0 (socket says 172.21.15.117:41374) 2020-11-20T13:56:23.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 -- 172.21.15.117:0/2039270356 learned_addr learned my addr 172.21.15.117:0/2039270356 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:23.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40acb71700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40a80be620 0x7f40a8149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40adb73700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a81473d0 0x7f40a815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 -- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a81473d0 msgr2=0x7f40a815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a81473d0 0x7f40a815a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 -- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40a80be620 msgr2=0x7f40a8149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40a80be620 0x7f40a8149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.547+0000 7f40ad372700 1 -- 172.21.15.117:0/2039270356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8092810 con 0x7f40a8067b40 2020-11-20T13:56:23.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.548+0000 7f40ad372700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a8067b40 0x7f40a8091800 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f40a80b8330 tx=0x7f4098007b50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.548+0000 7f40a67fc700 1 -- 172.21.15.117:0/2039270356 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4098007030 con 0x7f40a8067b40 2020-11-20T13:56:23.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.548+0000 7f40a67fc700 1 -- 172.21.15.117:0/2039270356 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098007190 con 0x7f40a8067b40 2020-11-20T13:56:23.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.548+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a8090070 con 0x7f40a8067b40 2020-11-20T13:56:23.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.548+0000 7f40a67fc700 1 -- 172.21.15.117:0/2039270356 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40980064e0 con 0x7f40a8067b40 2020-11-20T13:56:23.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.548+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a815a600 con 0x7f40a8067b40 2020-11-20T13:56:23.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.549+0000 7f40a67fc700 1 -- 172.21.15.117:0/2039270356 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4098010070 con 0x7f40a8067b40 2020-11-20T13:56:23.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.549+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a8092530 con 0x7f40a8067b40 2020-11-20T13:56:23.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.549+0000 7f40a67fc700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40880513c0 0x7f4088053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.549+0000 7f40a67fc700 1 -- 172.21.15.117:0/2039270356 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7f4098068610 con 0x7f40a8067b40 2020-11-20T13:56:23.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.549+0000 7f40acb71700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40880513c0 0x7f4088053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.549+0000 7f40acb71700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40880513c0 0x7f4088053880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f409c01c000 tx=0x7f409c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.552+0000 7f40a67fc700 1 -- 172.21.15.117:0/2039270356 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f40980407d0 con 0x7f40a8067b40 2020-11-20T13:56:23.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.752+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40a8057050 con 0x7f40a8067b40 2020-11-20T13:56:23.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.753+0000 7f40a67fc700 1 -- 172.21.15.117:0/2039270356 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1774 (secure 0 0 0) 0x7f4098018020 con 0x7f40a8067b40 2020-11-20T13:56:23.754 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:23.755 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/2183729070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":2183729070},{"type":"v1","addr":"172.21.15.85:6833","nonce":2183729070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:55:50.675265+0000","modified":"2020-11-20T13:56:23.416949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T13:56:23.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40880513c0 msgr2=0x7f4088053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40880513c0 0x7f4088053880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f409c01c000 tx=0x7f409c015000).stop 2020-11-20T13:56:23.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a8067b40 msgr2=0x7f40a8091800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a8067b40 0x7f40a8091800 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f40a80b8330 tx=0x7f4098007b50).stop 2020-11-20T13:56:23.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 shutdown_connections 2020-11-20T13:56:23.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f40880513c0 0x7f4088053880 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40a81473d0 0x7f40a815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40a8067b40 0x7f40a8091800 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 --2- 172.21.15.117:0/2039270356 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f40a80be620 0x7f40a8149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 >> 172.21.15.117:0/2039270356 conn(0x7f40a805c4c0 msgr2=0x7f40a8146300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:23.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 shutdown_connections 2020-11-20T13:56:23.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.754+0000 7f40af5d6700 1 -- 172.21.15.117:0/2039270356 wait complete. 2020-11-20T13:56:23.758 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 6 2020-11-20T13:56:23.766 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs rm cephfs --yes-i-really-mean-it 2020-11-20T13:56:23.917 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.917+0000 7fb6ee323700 1 Processor -- start 2020-11-20T13:56:23.918 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.917+0000 7fb6ee323700 1 -- start start 2020-11-20T13:56:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ee323700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb6e809bbb0 0x7fb6e8143660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.920 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ee323700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6e809aef0 0x7fb6e8143b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.921 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ee323700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e809b3c0 con 0x7fb6e80b4f20 2020-11-20T13:56:23.922 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ee323700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80ae560 con 0x7fb6e809bbb0 2020-11-20T13:56:23.922 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ee323700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e809bf90 con 0x7fb6e809aef0 2020-11-20T13:56:23.922 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ecb20700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6e809aef0 0x7fb6e8143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.922 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ecb20700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6e809aef0 0x7fb6e8143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41386/0 (socket says 172.21.15.117:41386) 2020-11-20T13:56:23.923 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ecb20700 1 -- 172.21.15.117:0/2131269012 learned_addr learned my addr 172.21.15.117:0/2131269012 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:23.923 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.918+0000 7fb6ed321700 1 --2- 172.21.15.117:0/2131269012 >> v2:172.21.15.85:3300/0 conn(0x7fb6e809bbb0 0x7fb6e8143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.923 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ecb20700 1 -- 172.21.15.117:0/2131269012 >> v2:172.21.15.85:3300/0 conn(0x7fb6e809bbb0 msgr2=0x7fb6e8143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:23.923 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ecb20700 1 --2- 172.21.15.117:0/2131269012 >> v2:172.21.15.85:3300/0 conn(0x7fb6e809bbb0 0x7fb6e8143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.924 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ecb20700 1 -- 172.21.15.117:0/2131269012 >> v1:172.21.15.85:6789/0 conn(0x7fb6e80b4f20 legacy=0x7fb6e805ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.924 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ecb20700 1 -- 172.21.15.117:0/2131269012 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8144040 con 0x7fb6e809aef0 2020-11-20T13:56:23.924 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ecb20700 1 --2- 172.21.15.117:0/2131269012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e809aef0 0x7fb6e8143b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fb6e4002f30 tx=0x7fb6e4006e80).ready entity=mon.1 client_cookie=a3e25c51b4af195a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.924 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6d7fff700 1 -- 172.21.15.117:0/2131269012 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e40097f0 con 0x7fb6e809aef0 2020-11-20T13:56:23.924 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6d7fff700 1 -- 172.21.15.117:0/2131269012 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e4009950 con 0x7fb6e809aef0 2020-11-20T13:56:23.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6d7fff700 1 -- 172.21.15.117:0/2131269012 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e4009cb0 con 0x7fb6e809aef0 2020-11-20T13:56:23.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 -- 172.21.15.117:0/2131269012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e809aef0 msgr2=0x7fb6e8143b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 --2- 172.21.15.117:0/2131269012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e809aef0 0x7fb6e8143b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fb6e4002f30 tx=0x7fb6e4006e80).stop 2020-11-20T13:56:23.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 -- 172.21.15.117:0/2131269012 shutdown_connections 2020-11-20T13:56:23.925 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 --2- 172.21.15.117:0/2131269012 >> v2:172.21.15.85:3300/0 conn(0x7fb6e809bbb0 0x7fb6e8143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.926 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 --2- 172.21.15.117:0/2131269012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e809aef0 0x7fb6e8143b50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.926 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 -- 172.21.15.117:0/2131269012 >> 172.21.15.117:0/2131269012 conn(0x7fb6e80b9470 msgr2=0x7fb6e80b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:23.926 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 -- 172.21.15.117:0/2131269012 shutdown_connections 2020-11-20T13:56:23.926 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.919+0000 7fb6ee323700 1 -- 172.21.15.117:0/2131269012 wait complete. 2020-11-20T13:56:23.927 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.920+0000 7fb6ee323700 1 Processor -- start 2020-11-20T13:56:23.927 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.920+0000 7fb6ee323700 1 -- start start 2020-11-20T13:56:23.929 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ee323700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb6e809aef0 0x7fb6e80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.929 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ee323700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb6e809bbb0 0x7fb6e80658d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.930 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ee323700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e80b4f20 0x7fb6e8159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.931 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ee323700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8065dc0 con 0x7fb6e809aef0 2020-11-20T13:56:23.932 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ee323700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e815a330 con 0x7fb6e80b4f20 2020-11-20T13:56:23.932 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ee323700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e815a5d0 con 0x7fb6e809bbb0 2020-11-20T13:56:23.932 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e80b4f20 0x7fb6e8159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.932 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e80b4f20 0x7fb6e8159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41392/0 (socket says 172.21.15.117:41392) 2020-11-20T13:56:23.933 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 -- 172.21.15.117:0/445201034 learned_addr learned my addr 172.21.15.117:0/445201034 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:23.933 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ecb20700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb6e809bbb0 0x7fb6e80658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.933 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6ed321700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb6e809aef0 0x7fb6e80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.933 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 -- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb6e809bbb0 msgr2=0x7fb6e80658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.933 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb6e809bbb0 0x7fb6e80658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.934 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 -- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb6e809aef0 msgr2=0x7fb6e80b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:23.934 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb6e809aef0 0x7fb6e80b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:23.934 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.921+0000 7fb6edb22700 1 -- 172.21.15.117:0/445201034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8159f80 con 0x7fb6e80b4f20 2020-11-20T13:56:23.934 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.922+0000 7fb6edb22700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e80b4f20 0x7fb6e8159a90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb6e809a4d0 tx=0x7fb6e0007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.935 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.922+0000 7fb6d67fc700 1 -- 172.21.15.117:0/445201034 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e0006860 con 0x7fb6e80b4f20 2020-11-20T13:56:23.935 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.922+0000 7fb6d67fc700 1 -- 172.21.15.117:0/445201034 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e00069c0 con 0x7fb6e80b4f20 2020-11-20T13:56:23.935 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.922+0000 7fb6d67fc700 1 -- 172.21.15.117:0/445201034 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6e0006d20 con 0x7fb6e80b4f20 2020-11-20T13:56:23.935 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.922+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e8066db0 con 0x7fb6e80b4f20 2020-11-20T13:56:23.936 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.922+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e8066350 con 0x7fb6e80b4f20 2020-11-20T13:56:23.936 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.923+0000 7fb6d67fc700 1 -- 172.21.15.117:0/445201034 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb6e0010070 con 0x7fb6e80b4f20 2020-11-20T13:56:23.936 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.923+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e8066f40 con 0x7fb6e80b4f20 2020-11-20T13:56:23.936 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.923+0000 7fb6d67fc700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb6c8051210 0x7fb6c80536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:23.936 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.923+0000 7fb6d67fc700 1 -- 172.21.15.117:0/445201034 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7fb6e00688b0 con 0x7fb6e80b4f20 2020-11-20T13:56:23.937 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.923+0000 7fb6ed321700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb6c8051210 0x7fb6c80536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:23.937 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.923+0000 7fb6ed321700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb6c8051210 0x7fb6c80536d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb6d8008000 tx=0x7fb6d8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:23.937 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:23.927+0000 7fb6d67fc700 1 -- 172.21.15.117:0/445201034 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb6e0040a70 con 0x7fb6e80b4f20 2020-11-20T13:56:24.126 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.126+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb6e8057050 con 0x7fb6e80b4f20 2020-11-20T13:56:24.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.424+0000 7fb6d67fc700 1 -- 172.21.15.117:0/445201034 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6e0044090 con 0x7fb6e80b4f20 2020-11-20T13:56:24.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb6c8051210 msgr2=0x7fb6c80536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb6c8051210 0x7fb6c80536d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb6d8008000 tx=0x7fb6d8008040).stop 2020-11-20T13:56:24.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e80b4f20 msgr2=0x7fb6e8159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e80b4f20 0x7fb6e8159a90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb6e809a4d0 tx=0x7fb6e0007eb0).stop 2020-11-20T13:56:24.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 shutdown_connections 2020-11-20T13:56:24.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb6c8051210 0x7fb6c80536d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6e80b4f20 0x7fb6e8159a90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb6e809aef0 0x7fb6e80b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 --2- 172.21.15.117:0/445201034 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb6e809bbb0 0x7fb6e80658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 >> 172.21.15.117:0/445201034 conn(0x7fb6e80b9470 msgr2=0x7fb6e809c500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:24.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 shutdown_connections 2020-11-20T13:56:24.429 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.426+0000 7fb6ee323700 1 -- 172.21.15.117:0/445201034 wait complete. 2020-11-20T13:56:24.437 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T13:56:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.547+0000 7feaef1d4700 1 Processor -- start 2020-11-20T13:56:24.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.547+0000 7feaef1d4700 1 -- start start 2020-11-20T13:56:24.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.547+0000 7feaef1d4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae8098d60 0x7feae809b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.547+0000 7feaef1d4700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feae80b5240 0x7feae80630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.547+0000 7feaef1d4700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae80b4860 con 0x7feae80635b0 2020-11-20T13:56:24.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.547+0000 7feaef1d4700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae81439f0 con 0x7feae80b5240 2020-11-20T13:56:24.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.547+0000 7feaef1d4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae80b4b00 con 0x7feae8098d60 2020-11-20T13:56:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feae80b5240 0x7feae80630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feae80b5240 0x7feae80630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41398/0 (socket says 172.21.15.117:41398) 2020-11-20T13:56:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 -- 172.21.15.117:0/2979624245 learned_addr learned my addr 172.21.15.117:0/2979624245 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feaecf70700 1 --2- 172.21.15.117:0/2979624245 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae8098d60 0x7feae809b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 -- 172.21.15.117:0/2979624245 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae8098d60 msgr2=0x7feae809b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:24.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 --2- 172.21.15.117:0/2979624245 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae8098d60 0x7feae809b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 -- 172.21.15.117:0/2979624245 >> v1:172.21.15.85:6789/0 conn(0x7feae80635b0 legacy=0x7feae809c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 -- 172.21.15.117:0/2979624245 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80b4da0 con 0x7feae80b5240 2020-11-20T13:56:24.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae7fff700 1 --2- 172.21.15.117:0/2979624245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae80b5240 0x7feae80630c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7feadc002f30 tx=0x7feadc006f10).ready entity=mon.1 client_cookie=42f56edaf2ad25f0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:24.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae77fe700 1 -- 172.21.15.117:0/2979624245 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feadc00cb70 con 0x7feae80b5240 2020-11-20T13:56:24.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae77fe700 1 -- 172.21.15.117:0/2979624245 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc00ccd0 con 0x7feae80b5240 2020-11-20T13:56:24.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feae77fe700 1 -- 172.21.15.117:0/2979624245 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feadc010600 con 0x7feae80b5240 2020-11-20T13:56:24.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feaef1d4700 1 -- 172.21.15.117:0/2979624245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae80b5240 msgr2=0x7feae80630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.548+0000 7feaef1d4700 1 --2- 172.21.15.117:0/2979624245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae80b5240 0x7feae80630c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7feadc002f30 tx=0x7feadc006f10).stop 2020-11-20T13:56:24.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.549+0000 7feaef1d4700 1 -- 172.21.15.117:0/2979624245 shutdown_connections 2020-11-20T13:56:24.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.549+0000 7feaef1d4700 1 --2- 172.21.15.117:0/2979624245 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae8098d60 0x7feae809b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.549+0000 7feaef1d4700 1 --2- 172.21.15.117:0/2979624245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae80b5240 0x7feae80630c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.549+0000 7feaef1d4700 1 -- 172.21.15.117:0/2979624245 >> 172.21.15.117:0/2979624245 conn(0x7feae80b9770 msgr2=0x7feae80b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:24.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.549+0000 7feaef1d4700 1 -- 172.21.15.117:0/2979624245 shutdown_connections 2020-11-20T13:56:24.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.549+0000 7feaef1d4700 1 -- 172.21.15.117:0/2979624245 wait complete. 2020-11-20T13:56:24.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 Processor -- start 2020-11-20T13:56:24.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 -- start start 2020-11-20T13:56:24.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae80635b0 0x7feae80983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae8098d60 0x7feae80b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feae80b5240 0x7feae8064ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae808fc60 con 0x7feae80b5240 2020-11-20T13:56:24.561 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae808ff30 con 0x7feae8098d60 2020-11-20T13:56:24.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feaef1d4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae8090160 con 0x7feae80635b0 2020-11-20T13:56:24.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feae7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae8098d60 0x7feae80b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.550+0000 7feae7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae8098d60 0x7feae80b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41404/0 (socket says 172.21.15.117:41404) 2020-11-20T13:56:24.562 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae7fff700 1 -- 172.21.15.117:0/803417380 learned_addr learned my addr 172.21.15.117:0/803417380 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:24.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feaed771700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feae80b5240 0x7feae8064ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feaecf70700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae80635b0 0x7feae80983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae7fff700 1 -- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae80635b0 msgr2=0x7feae80983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae7fff700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae80635b0 0x7feae80983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.563 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae7fff700 1 -- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feae80b5240 msgr2=0x7feae8064ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:24.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae7fff700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feae80b5240 0x7feae8064ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:24.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae7fff700 1 -- 172.21.15.117:0/803417380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8090400 con 0x7feae8098d60 2020-11-20T13:56:24.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae7fff700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae8098d60 0x7feae80b2e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7feae808fda0 tx=0x7feadc006f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:24.564 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae5ffb700 1 -- 172.21.15.117:0/803417380 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feadc01a840 con 0x7feae8098d60 2020-11-20T13:56:24.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae5ffb700 1 -- 172.21.15.117:0/803417380 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc01a9a0 con 0x7feae8098d60 2020-11-20T13:56:24.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feae5ffb700 1 -- 172.21.15.117:0/803417380 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feadc01ad00 con 0x7feae8098d60 2020-11-20T13:56:24.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae80906a0 con 0x7feae8098d60 2020-11-20T13:56:24.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.551+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae8091010 con 0x7feae8098d60 2020-11-20T13:56:24.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.552+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae8057050 con 0x7feae8098d60 2020-11-20T13:56:24.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.552+0000 7feae5ffb700 1 -- 172.21.15.117:0/803417380 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7feadc02a070 con 0x7feae8098d60 2020-11-20T13:56:24.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.552+0000 7feae5ffb700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feac80513c0 0x7feac8053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:24.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.552+0000 7feae5ffb700 1 -- 172.21.15.117:0/803417380 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6517+0+0 (secure 0 0 0) 0x7feadc06aed0 con 0x7feae8098d60 2020-11-20T13:56:24.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.552+0000 7feaecf70700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feac80513c0 0x7feac8053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:24.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.553+0000 7feaecf70700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feac80513c0 0x7feac8053880 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fead800b000 tx=0x7fead8006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:24.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.556+0000 7feae5ffb700 1 -- 172.21.15.117:0/803417380 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7feadc043090 con 0x7feae8098d60 2020-11-20T13:56:24.706 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:24.705+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feae805ca00 con 0x7feae8098d60 2020-11-20T13:56:25.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.466+0000 7feae5ffb700 1 -- 172.21.15.117:0/803417380 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 185+0+0 (secure 0 0 0) 0x7feadc0466b0 con 0x7feae8098d60 2020-11-20T13:56:25.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.468+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feac80513c0 msgr2=0x7feac8053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.468+0000 7feaef1d4700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feac80513c0 0x7feac8053880 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fead800b000 tx=0x7fead8006dc0).stop 2020-11-20T13:56:25.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.468+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae8098d60 msgr2=0x7feae80b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.468+0000 7feaef1d4700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae8098d60 0x7feae80b2e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7feae808fda0 tx=0x7feadc006f10).stop 2020-11-20T13:56:25.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 shutdown_connections 2020-11-20T13:56:25.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7feac80513c0 0x7feac8053880 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7feae80635b0 0x7feae80983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7feae80b5240 0x7feae8064ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 --2- 172.21.15.117:0/803417380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feae8098d60 0x7feae80b2e60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 >> 172.21.15.117:0/803417380 conn(0x7feae80b9770 msgr2=0x7feae8062750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 shutdown_connections 2020-11-20T13:56:25.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.469+0000 7feaef1d4700 1 -- 172.21.15.117:0/803417380 wait complete. 2020-11-20T13:56:25.472 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' removed 2020-11-20T13:56:25.483 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T13:56:25.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.598+0000 7fd14a698700 1 Processor -- start 2020-11-20T13:56:25.599 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.598+0000 7fd14a698700 1 -- start start 2020-11-20T13:56:25.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd144062910 0x7fd14409aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd144062cf0 0x7fd14405f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd14409af30 0x7fd14405fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14409b3c0 con 0x7fd144062910 2020-11-20T13:56:25.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14409b660 con 0x7fd14409af30 2020-11-20T13:56:25.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14409b900 con 0x7fd144062cf0 2020-11-20T13:56:25.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd14409af30 0x7fd14405fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd14409af30 0x7fd14405fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41414/0 (socket says 172.21.15.117:41414) 2020-11-20T13:56:25.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 -- 172.21.15.117:0/3817263514 learned_addr learned my addr 172.21.15.117:0/3817263514 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:25.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd1437fe700 1 --2- 172.21.15.117:0/3817263514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd144062cf0 0x7fd14405f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd143fff700 1 --2- 172.21.15.117:0/3817263514 >> v2:172.21.15.85:3300/0 conn(0x7fd144062910 0x7fd14409aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 -- 172.21.15.117:0/3817263514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd144062cf0 msgr2=0x7fd14405f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:25.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 --2- 172.21.15.117:0/3817263514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd144062cf0 0x7fd14405f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 -- 172.21.15.117:0/3817263514 >> v2:172.21.15.85:3300/0 conn(0x7fd144062910 msgr2=0x7fd14409aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:25.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 --2- 172.21.15.117:0/3817263514 >> v2:172.21.15.85:3300/0 conn(0x7fd144062910 0x7fd14409aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 -- 172.21.15.117:0/3817263514 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1440b4310 con 0x7fd14409af30 2020-11-20T13:56:25.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd143fff700 1 --2- 172.21.15.117:0/3817263514 >> v2:172.21.15.85:3300/0 conn(0x7fd144062910 0x7fd14409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:25.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd148c35700 1 --2- 172.21.15.117:0/3817263514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd14409af30 0x7fd14405fe00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd13c0032b0 tx=0x7fd13c009270).ready entity=mon.1 client_cookie=de71a646f4f3da2f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd142ffd700 1 -- 172.21.15.117:0/3817263514 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd13c009e60 con 0x7fd14409af30 2020-11-20T13:56:25.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd142ffd700 1 -- 172.21.15.117:0/3817263514 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c00bc00 con 0x7fd14409af30 2020-11-20T13:56:25.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd142ffd700 1 -- 172.21.15.117:0/3817263514 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd13c00bd70 con 0x7fd14409af30 2020-11-20T13:56:25.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 -- 172.21.15.117:0/3817263514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd14409af30 msgr2=0x7fd14405fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.599+0000 7fd14a698700 1 --2- 172.21.15.117:0/3817263514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd14409af30 0x7fd14405fe00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd13c0032b0 tx=0x7fd13c009270).stop 2020-11-20T13:56:25.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.600+0000 7fd14a698700 1 -- 172.21.15.117:0/3817263514 shutdown_connections 2020-11-20T13:56:25.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.600+0000 7fd14a698700 1 --2- 172.21.15.117:0/3817263514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd14409af30 0x7fd14405fe00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.600+0000 7fd14a698700 1 --2- 172.21.15.117:0/3817263514 >> v2:172.21.15.85:3300/0 conn(0x7fd144062910 0x7fd14409aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.600+0000 7fd14a698700 1 --2- 172.21.15.117:0/3817263514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd144062cf0 0x7fd14405f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.600+0000 7fd14a698700 1 -- 172.21.15.117:0/3817263514 >> 172.21.15.117:0/3817263514 conn(0x7fd1440b9770 msgr2=0x7fd1440b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:25.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.600+0000 7fd14a698700 1 -- 172.21.15.117:0/3817263514 shutdown_connections 2020-11-20T13:56:25.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.600+0000 7fd14a698700 1 -- 172.21.15.117:0/3817263514 wait complete. 2020-11-20T13:56:25.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.601+0000 7fd14a698700 1 Processor -- start 2020-11-20T13:56:25.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.601+0000 7fd14a698700 1 -- start start 2020-11-20T13:56:25.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd144062910 0x7fd14414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd144062cf0 0x7fd14405fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd14415dce0 0x7fd144067280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14414daf0 con 0x7fd144062910 2020-11-20T13:56:25.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14414dd90 con 0x7fd144062cf0 2020-11-20T13:56:25.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14414e030 con 0x7fd14415dce0 2020-11-20T13:56:25.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd144062cf0 0x7fd14405fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd144062cf0 0x7fd14405fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41418/0 (socket says 172.21.15.117:41418) 2020-11-20T13:56:25.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 -- 172.21.15.117:0/2383873234 learned_addr learned my addr 172.21.15.117:0/2383873234 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:25.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd143fff700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd144062910 0x7fd14414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.614 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd148c35700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd14415dce0 0x7fd144067280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 -- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd14415dce0 msgr2=0x7fd144067280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd14415dce0 0x7fd144067280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 -- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd144062910 msgr2=0x7fd14414e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:25.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd144062910 0x7fd14414e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:25.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 -- 172.21.15.117:0/2383873234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14414d700 con 0x7fd144062cf0 2020-11-20T13:56:25.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd148c35700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd14415dce0 0x7fd144067280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:25.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1437fe700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd144062cf0 0x7fd14405fc10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd13800c000 tx=0x7fd138006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.616 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1417fa700 1 -- 172.21.15.117:0/2383873234 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd13800ca10 con 0x7fd144062cf0 2020-11-20T13:56:25.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1417fa700 1 -- 172.21.15.117:0/2383873234 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13800cb70 con 0x7fd144062cf0 2020-11-20T13:56:25.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd144065d00 con 0x7fd144062cf0 2020-11-20T13:56:25.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd1417fa700 1 -- 172.21.15.117:0/2383873234 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1380123f0 con 0x7fd144062cf0 2020-11-20T13:56:25.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.602+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1440679b0 con 0x7fd144062cf0 2020-11-20T13:56:25.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.603+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14414d950 con 0x7fd144062cf0 2020-11-20T13:56:25.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.603+0000 7fd1417fa700 1 -- 172.21.15.117:0/2383873234 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd138012550 con 0x7fd144062cf0 2020-11-20T13:56:25.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.604+0000 7fd1417fa700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd124051320 0x7fd1240537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:25.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.604+0000 7fd1417fa700 1 -- 172.21.15.117:0/2383873234 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6079+0+0 (secure 0 0 0) 0x7fd138069440 con 0x7fd144062cf0 2020-11-20T13:56:25.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.604+0000 7fd143fff700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd124051320 0x7fd1240537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:25.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.604+0000 7fd143fff700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd124051320 0x7fd1240537e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fd13400c000 tx=0x7fd134008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:25.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.607+0000 7fd1417fa700 1 -- 172.21.15.117:0/2383873234 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd14414d950 con 0x7fd144062cf0 2020-11-20T13:56:25.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:25.756+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd14414d950 con 0x7fd144062cf0 2020-11-20T13:56:26.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.470+0000 7fd1417fa700 1 -- 172.21.15.117:0/2383873234 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd14414d950 con 0x7fd144062cf0 2020-11-20T13:56:26.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd124051320 msgr2=0x7fd1240537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd124051320 0x7fd1240537e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fd13400c000 tx=0x7fd134008040).stop 2020-11-20T13:56:26.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd144062cf0 msgr2=0x7fd14405fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd144062cf0 0x7fd14405fc10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fd13800c000 tx=0x7fd138006dc0).stop 2020-11-20T13:56:26.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 shutdown_connections 2020-11-20T13:56:26.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd124051320 0x7fd1240537e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd14415dce0 0x7fd144067280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd144062910 0x7fd14414e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 --2- 172.21.15.117:0/2383873234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd144062cf0 0x7fd14405fc10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.472+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 >> 172.21.15.117:0/2383873234 conn(0x7fd1440b9770 msgr2=0x7fd1440b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:26.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.473+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 shutdown_connections 2020-11-20T13:56:26.477 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.473+0000 7fd14a698700 1 -- 172.21.15.117:0/2383873234 wait complete. 2020-11-20T13:56:26.478 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' removed 2020-11-20T13:56:26.484 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T13:56:26.485 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T13:56:26.485 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T13:56:26.485 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T13:56:26.486 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T13:56:26.486 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T13:56:26.486 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T13:56:26.486 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T13:56:26.494 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T13:56:26.495 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T13:56:26.496 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T13:56:26.496 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T13:56:26.539 INFO:tasks.ceph.mds.a:Started 2020-11-20T13:56:26.540 INFO:tasks.ceph.mds.b:Started 2020-11-20T13:56:26.540 DEBUG:teuthology.parallel:result is None 2020-11-20T13:56:26.541 DEBUG:teuthology.parallel:result is None 2020-11-20T13:56:26.541 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd blacklist clear 2020-11-20T13:56:26.617 INFO:tasks.ceph.mds.a.smithi117.stdout:starting mds.a at 2020-11-20T13:56:26.624 INFO:tasks.ceph.mds.b.smithi085.stdout:starting mds.b at 2020-11-20T13:56:26.661 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.654+0000 7f9dc68f8700 1 Processor -- start 2020-11-20T13:56:26.661 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.654+0000 7f9dc68f8700 1 -- start start 2020-11-20T13:56:26.662 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc68f8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9dc0067b40 0x7f9dc00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.662 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc68f8700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc00b4a00 con 0x7f9dc00be620 2020-11-20T13:56:26.662 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc68f8700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc014af20 con 0x7f9dc01473d0 2020-11-20T13:56:26.662 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc68f8700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc014b1c0 con 0x7f9dc0067b40 2020-11-20T13:56:26.662 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc4e95700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9dc0067b40 0x7f9dc00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.663 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc4e95700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9dc0067b40 0x7f9dc00b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41440/0 (socket says 172.21.15.117:41440) 2020-11-20T13:56:26.663 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dbf7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9dc01473d0 0x7f9dc00b8530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60598/0 (socket says 172.21.15.117:60598) 2020-11-20T13:56:26.663 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc4e95700 1 -- 172.21.15.117:0/86702745 learned_addr learned my addr 172.21.15.117:0/86702745 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:26.663 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc4e95700 1 -- 172.21.15.117:0/86702745 >> v1:172.21.15.117:6789/0 conn(0x7f9dc01473d0 legacy=0x7f9dc00b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc4e95700 1 -- 172.21.15.117:0/86702745 >> v1:172.21.15.85:6789/0 conn(0x7f9dc00be620 legacy=0x7f9dc0146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc4e95700 1 -- 172.21.15.117:0/86702745 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc014b460 con 0x7f9dc0067b40 2020-11-20T13:56:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.655+0000 7f9dc4e95700 1 --2- 172.21.15.117:0/86702745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc0067b40 0x7f9dc00b8c40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f9db80032b0 tx=0x7f9db8009270).ready entity=mon.1 client_cookie=7452f7d314ce5b8e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dbeffd700 1 -- 172.21.15.117:0/86702745 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9db800bc00 con 0x7f9dc0067b40 2020-11-20T13:56:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dbeffd700 1 -- 172.21.15.117:0/86702745 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db800bd60 con 0x7f9dc0067b40 2020-11-20T13:56:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dbeffd700 1 -- 172.21.15.117:0/86702745 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9db8011400 con 0x7f9dc0067b40 2020-11-20T13:56:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dc68f8700 1 -- 172.21.15.117:0/86702745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc0067b40 msgr2=0x7f9dc00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/86702745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc0067b40 0x7f9dc00b8c40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f9db80032b0 tx=0x7f9db8009270).stop 2020-11-20T13:56:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dc68f8700 1 -- 172.21.15.117:0/86702745 shutdown_connections 2020-11-20T13:56:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/86702745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc0067b40 0x7f9dc00b8c40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dc68f8700 1 -- 172.21.15.117:0/86702745 >> 172.21.15.117:0/86702745 conn(0x7f9dc005c4c0 msgr2=0x7f9dc00bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dc68f8700 1 -- 172.21.15.117:0/86702745 shutdown_connections 2020-11-20T13:56:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.656+0000 7f9dc68f8700 1 -- 172.21.15.117:0/86702745 wait complete. 2020-11-20T13:56:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.657+0000 7f9dc68f8700 1 Processor -- start 2020-11-20T13:56:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.657+0000 7f9dc68f8700 1 -- start start 2020-11-20T13:56:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dc68f8700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9dc0067b40 0x7f9dc00b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dc68f8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc00be620 0x7f9dc00a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dc68f8700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9dc01473d0 0x7f9dc0157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dc68f8700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0156a10 con 0x7f9dc01473d0 2020-11-20T13:56:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dc68f8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0156c00 con 0x7f9dc00be620 2020-11-20T13:56:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dc68f8700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0156ea0 con 0x7f9dc0067b40 2020-11-20T13:56:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc00be620 0x7f9dc00a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc00be620 0x7f9dc00a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41444/0 (socket says 172.21.15.117:41444) 2020-11-20T13:56:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 -- 172.21.15.117:0/2179407253 learned_addr learned my addr 172.21.15.117:0/2179407253 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbffff700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9dc0067b40 0x7f9dc00b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dc4e95700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9dc01473d0 0x7f9dc0157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 -- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9dc0067b40 msgr2=0x7f9dc00b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9dc0067b40 0x7f9dc00b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 -- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9dc01473d0 msgr2=0x7f9dc0157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:26.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9dc01473d0 0x7f9dc0157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:26.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 -- 172.21.15.117:0/2179407253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00a2a90 con 0x7f9dc00be620 2020-11-20T13:56:26.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.658+0000 7f9dbf7fe700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc00be620 0x7f9dc00a25a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f9db4000fe0 tx=0x7f9db4007be0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:26.670 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.659+0000 7f9dbd7fa700 1 -- 172.21.15.117:0/2179407253 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9db40072e0 con 0x7f9dc00be620 2020-11-20T13:56:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.659+0000 7f9dbd7fa700 1 -- 172.21.15.117:0/2179407253 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db4005520 con 0x7f9dc00be620 2020-11-20T13:56:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.659+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc0157830 con 0x7f9dc00be620 2020-11-20T13:56:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.659+0000 7f9dbd7fa700 1 -- 172.21.15.117:0/2179407253 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9db4005880 con 0x7f9dc00be620 2020-11-20T13:56:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.659+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc008fa90 con 0x7f9dc00be620 2020-11-20T13:56:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.660+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0157060 con 0x7f9dc00be620 2020-11-20T13:56:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.660+0000 7f9dbd7fa700 1 -- 172.21.15.117:0/2179407253 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9db4010070 con 0x7f9dc00be620 2020-11-20T13:56:26.673 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.660+0000 7f9dbd7fa700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9da0051290 0x7f9da0053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:26.673 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.660+0000 7f9dbd7fa700 1 -- 172.21.15.117:0/2179407253 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5686+0+0 (secure 0 0 0) 0x7f9db40682f0 con 0x7f9dc00be620 2020-11-20T13:56:26.673 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.660+0000 7f9dbffff700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9da0051290 0x7f9da0053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:26.673 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.660+0000 7f9dbffff700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9da0051290 0x7f9da0053750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9db000d000 tx=0x7f9db0007ad0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:26.674 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.663+0000 7f9dbd7fa700 1 -- 172.21.15.117:0/2179407253 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9db40408a0 con 0x7f9dc00be620 2020-11-20T13:56:26.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:26.828+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f9dc0057050 con 0x7f9dc00be620 2020-11-20T13:56:27.641 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.641+0000 7f9dbd7fa700 1 -- 172.21.15.117:0/2179407253 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9db4043ec0 con 0x7f9dc00be620 2020-11-20T13:56:27.643 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9da0051290 msgr2=0x7f9da0053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9da0051290 0x7f9da0053750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9db000d000 tx=0x7f9db0007ad0).stop 2020-11-20T13:56:27.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc00be620 msgr2=0x7f9dc00a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc00be620 0x7f9dc00a25a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f9db4000fe0 tx=0x7f9db4007be0).stop 2020-11-20T13:56:27.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 shutdown_connections 2020-11-20T13:56:27.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9da0051290 0x7f9da0053750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9dc0067b40 0x7f9dc00b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9dc01473d0 0x7f9dc0157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 --2- 172.21.15.117:0/2179407253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc00be620 0x7f9dc00a25a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 >> 172.21.15.117:0/2179407253 conn(0x7f9dc005c4c0 msgr2=0x7f9dc00ba600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:27.646 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 shutdown_connections 2020-11-20T13:56:27.646 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.643+0000 7f9dc68f8700 1 -- 172.21.15.117:0/2179407253 wait complete. 2020-11-20T13:56:27.646 INFO:teuthology.orchestra.run.smithi117.stderr: removed all blacklist entries 2020-11-20T13:56:27.654 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2', '3'] 2020-11-20T13:56:27.655 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth ls --format=json-pretty 2020-11-20T13:56:27.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.765+0000 7ffbeb15d700 1 Processor -- start 2020-11-20T13:56:27.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbeb15d700 1 -- start start 2020-11-20T13:56:27.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbeb15d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffbe414c2b0 0x7ffbe414e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.769 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbeb15d700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe40b4e80 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.769 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbeb15d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe40b5120 con 0x7ffbe40b5860 2020-11-20T13:56:27.769 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbeb15d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40b53c0 con 0x7ffbe414c2b0 2020-11-20T13:56:27.770 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbe96fa700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffbe414c2b0 0x7ffbe414e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.770 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbe3fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ffbe40b5860 0x7ffbe4098b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60612/0 (socket says 172.21.15.117:60612) 2020-11-20T13:56:27.771 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbe3fff700 1 -- 172.21.15.117:0/1350666434 learned_addr learned my addr 172.21.15.117:0/1350666434 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:27.771 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.766+0000 7ffbe96fa700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffbe414c2b0 0x7ffbe414e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41452/0 (socket says 172.21.15.117:41452) 2020-11-20T13:56:27.771 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe96fa700 1 -- 172.21.15.117:0/1350666434 >> v1:172.21.15.117:6789/0 conn(0x7ffbe40b5860 legacy=0x7ffbe4098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.771 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe37fe700 1 -- 172.21.15.117:0/1350666434 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219202826 0 0) 0x7ffbe40b5120 con 0x7ffbe40b5860 2020-11-20T13:56:27.771 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe96fa700 1 -- 172.21.15.117:0/1350666434 >> v1:172.21.15.85:6789/0 conn(0x7ffbe40ae8b0 legacy=0x7ffbe40aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.772 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe96fa700 1 -- 172.21.15.117:0/1350666434 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe40b4ac0 con 0x7ffbe414c2b0 2020-11-20T13:56:27.772 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe96fa700 1 --2- 172.21.15.117:0/1350666434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe414c2b0 0x7ffbe414e6a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ffbdc0032b0 tx=0x7ffbdc0091a0).ready entity=mon.1 client_cookie=4fd57daa5305eb46 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.772 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe37fe700 1 -- 172.21.15.117:0/1350666434 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbdc00bc40 con 0x7ffbe414c2b0 2020-11-20T13:56:27.772 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe37fe700 1 -- 172.21.15.117:0/1350666434 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc00bda0 con 0x7ffbe414c2b0 2020-11-20T13:56:27.772 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbe37fe700 1 -- 172.21.15.117:0/1350666434 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbdc010600 con 0x7ffbe414c2b0 2020-11-20T13:56:27.773 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbeb15d700 1 -- 172.21.15.117:0/1350666434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe414c2b0 msgr2=0x7ffbe414e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.773 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/1350666434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe414c2b0 0x7ffbe414e6a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7ffbdc0032b0 tx=0x7ffbdc0091a0).stop 2020-11-20T13:56:27.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbeb15d700 1 -- 172.21.15.117:0/1350666434 shutdown_connections 2020-11-20T13:56:27.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/1350666434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe414c2b0 0x7ffbe414e6a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.767+0000 7ffbeb15d700 1 -- 172.21.15.117:0/1350666434 >> 172.21.15.117:0/1350666434 conn(0x7ffbe405b9f0 msgr2=0x7ffbe405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:27.774 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.768+0000 7ffbeb15d700 1 -- 172.21.15.117:0/1350666434 shutdown_connections 2020-11-20T13:56:27.775 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.768+0000 7ffbeb15d700 1 -- 172.21.15.117:0/1350666434 wait complete. 2020-11-20T13:56:27.778 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.768+0000 7ffbeb15d700 1 Processor -- start 2020-11-20T13:56:27.779 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.768+0000 7ffbeb15d700 1 -- start start 2020-11-20T13:56:27.779 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbeb15d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe40ae8b0 0x7ffbe40634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.780 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbeb15d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ffbe40b5860 0x7ffbe40b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.780 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbeb15d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ffbe414c2b0 0x7ffbe4064820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.780 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbeb15d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe4149c50 con 0x7ffbe40b5860 2020-11-20T13:56:27.780 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbeb15d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe4149ef0 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.781 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbeb15d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe414a190 con 0x7ffbe414c2b0 2020-11-20T13:56:27.781 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbe8ef9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe40ae8b0 0x7ffbe40634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.781 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbe8ef9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe40ae8b0 0x7ffbe40634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41456/0 (socket says 172.21.15.117:41456) 2020-11-20T13:56:27.781 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.769+0000 7ffbe8ef9700 1 -- 172.21.15.117:0/2711326896 learned_addr learned my addr 172.21.15.117:0/2711326896 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:27.782 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe96fa700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ffbe414c2b0 0x7ffbe4064820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.782 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe3fff700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ffbe40b5860 0x7ffbe40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.782 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe8ef9700 1 -- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ffbe414c2b0 msgr2=0x7ffbe4064820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.782 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe8ef9700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ffbe414c2b0 0x7ffbe4064820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.782 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe8ef9700 1 -- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ffbe40b5860 msgr2=0x7ffbe40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:27.783 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe8ef9700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ffbe40b5860 0x7ffbe40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:27.783 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe8ef9700 1 -- 172.21.15.117:0/2711326896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe4064460 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.783 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe3fff700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ffbe40b5860 0x7ffbe40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:27.783 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe8ef9700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe40ae8b0 0x7ffbe40634b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7ffbe409b040 tx=0x7ffbd4007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.783 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe1ffb700 1 -- 172.21.15.117:0/2711326896 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd400e4a0 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.784 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe1ffb700 1 -- 172.21.15.117:0/2711326896 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd400e600 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.784 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbe1ffb700 1 -- 172.21.15.117:0/2711326896 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffbd400e960 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.784 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe4149820 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.784 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.770+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe4064f50 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.784 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.771+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe40645f0 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.785 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.771+0000 7ffbe1ffb700 1 -- 172.21.15.117:0/2711326896 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ffbd4019030 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.785 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.771+0000 7ffbe1ffb700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ffbc4051210 0x7ffbc40536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:27.785 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.771+0000 7ffbe1ffb700 1 -- 172.21.15.117:0/2711326896 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7ffbd40681b0 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.785 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.771+0000 7ffbe3fff700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ffbc4051210 0x7ffbc40536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:27.785 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.772+0000 7ffbe3fff700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ffbc4051210 0x7ffbc40536d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ffbd800b000 tx=0x7ffbd8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:27.786 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.775+0000 7ffbe1ffb700 1 -- 172.21.15.117:0/2711326896 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ffbd40406b0 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.981 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.981+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffbe4057050 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.982 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.981+0000 7ffbe1ffb700 1 -- 172.21.15.117:0/2711326896 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+5917 (secure 0 0 0) 0x7ffbd4017020 con 0x7ffbe40ae8b0 2020-11-20T13:56:27.982 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:27.982 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2020-11-20T13:56:27.983 INFO:teuthology.orchestra.run.smithi117.stdout: "auth_dump": [ 2020-11-20T13:56:27.983 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.983 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "mds.a", 2020-11-20T13:56:27.983 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfEKfQIBAABP6nMcAOErqsnKUF6ZSfsg==", 2020-11-20T13:56:27.983 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.984 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow", 2020-11-20T13:56:27.984 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.984 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.984 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.984 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:27.985 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.985 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "mds.b", 2020-11-20T13:56:27.985 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfwdmmJRAAKx2MfVYCS9Q3q4qz47XB7Q==", 2020-11-20T13:56:27.986 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.986 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow", 2020-11-20T13:56:27.986 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.986 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.986 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.987 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:27.987 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.987 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.0", 2020-11-20T13:56:27.987 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCwyrdfY2R8MhAAVs+5CcEQIMFK/paz/FYP/A==", 2020-11-20T13:56:27.988 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.988 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.988 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.989 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.989 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:27.989 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.990 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.1", 2020-11-20T13:56:27.990 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCzyrdf82K1FBAAlG7E19KJBgdvlws3jqAX7Q==", 2020-11-20T13:56:27.991 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.991 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.992 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.992 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.992 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:27.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.993 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.2", 2020-11-20T13:56:27.993 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQC1yrdf2+NXMBAAsjWqWp3/hIRz2ZW5N6GBQg==", 2020-11-20T13:56:27.993 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.993 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.994 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.994 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:27.994 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.995 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.3", 2020-11-20T13:56:27.995 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQC4yrdf+pcsEBAAcq3g4vKVejtTK4xPJCnHQw==", 2020-11-20T13:56:27.995 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.995 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.995 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.996 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.996 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:27.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.996 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.4", 2020-11-20T13:56:27.997 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQC9yrdfbLG4IhAA8BapeiLCGll6S8dzis39OQ==", 2020-11-20T13:56:27.997 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.997 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.997 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.997 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.998 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:27.998 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:27.998 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.5", 2020-11-20T13:56:27.998 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDAyrdfixR6BRAAaztS0ZrTIwuMvnazHISWuQ==", 2020-11-20T13:56:27.999 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:27.999 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:27.999 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:27.999 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:27.999 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:27.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.000 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.000 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.6", 2020-11-20T13:56:28.000 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDCyrdffY00IRAA7l+trbmTc2xXBAOQ3mjFcA==", 2020-11-20T13:56:28.000 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.001 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:28.001 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:28.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:28.002 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.002 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.003 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "osd.7", 2020-11-20T13:56:28.003 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDFyrdf17WiARAAUfSZHp54qwZ98EB559uhqw==", 2020-11-20T13:56:28.003 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.003 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:28.003 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:28.004 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:28.004 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.004 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.004 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.0", 2020-11-20T13:56:28.005 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfQ6vdKRAAAom8Z8dJNDxlDSnrjb9q2g==", 2020-11-20T13:56:28.005 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.005 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow", 2020-11-20T13:56:28.005 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow r", 2020-11-20T13:56:28.005 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow rw", 2020-11-20T13:56:28.006 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow rwx" 2020-11-20T13:56:28.006 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.006 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.006 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.1", 2020-11-20T13:56:28.007 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfp8qALxAAIklppc7VXYVKg8gwnCc5ww==", 2020-11-20T13:56:28.007 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.007 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow", 2020-11-20T13:56:28.007 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow r", 2020-11-20T13:56:28.007 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow rw", 2020-11-20T13:56:28.008 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow rwx" 2020-11-20T13:56:28.008 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.008 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.2", 2020-11-20T13:56:28.009 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfmQrDMhAAm5c++eZqE35+xdgSAY8SPg==", 2020-11-20T13:56:28.009 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.009 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow", 2020-11-20T13:56:28.009 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow r", 2020-11-20T13:56:28.009 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow rw", 2020-11-20T13:56:28.010 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow rwx" 2020-11-20T13:56:28.010 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.010 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.013 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.3", 2020-11-20T13:56:28.013 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfUINnOBAACZQKMu1Q/kkZKb7j0qhM9A==", 2020-11-20T13:56:28.013 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.014 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow", 2020-11-20T13:56:28.014 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow r", 2020-11-20T13:56:28.014 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow rw", 2020-11-20T13:56:28.014 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow rwx" 2020-11-20T13:56:28.015 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.015 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.015 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.admin", 2020-11-20T13:56:28.015 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfV0IXARAAJQWE8qRBU3ddN9tMP6eHAw==", 2020-11-20T13:56:28.016 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.016 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow *", 2020-11-20T13:56:28.016 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr": "allow *", 2020-11-20T13:56:28.016 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow *", 2020-11-20T13:56:28.017 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:28.017 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.017 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.017 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.bootstrap-mds", 2020-11-20T13:56:28.018 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDSyrdf1WrkCxAAdySE+IuZqKf6at5egFnsgA==", 2020-11-20T13:56:28.018 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.018 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile bootstrap-mds" 2020-11-20T13:56:28.018 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.019 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.bootstrap-mgr", 2020-11-20T13:56:28.019 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDSyrdfXHfkCxAAv6Zjx7dtWcFJOsvSV4EWiw==", 2020-11-20T13:56:28.020 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.020 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile bootstrap-mgr" 2020-11-20T13:56:28.020 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.020 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.020 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.bootstrap-osd", 2020-11-20T13:56:28.021 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDSyrdfbX/kCxAAuAJ+JuJMg6b8WS4YVryz9g==", 2020-11-20T13:56:28.021 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.021 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile bootstrap-osd" 2020-11-20T13:56:28.021 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.022 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.022 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.bootstrap-rbd", 2020-11-20T13:56:28.022 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDSyrdfqojkCxAAALVaSfeFXVESe6q/9dnWQQ==", 2020-11-20T13:56:28.022 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.022 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile bootstrap-rbd" 2020-11-20T13:56:28.023 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.024 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.024 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.bootstrap-rbd-mirror", 2020-11-20T13:56:28.024 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDSyrdfZpHkCxAATMy+3Tj+FNkfLwMG/7hL4A==", 2020-11-20T13:56:28.024 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.024 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2020-11-20T13:56:28.025 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.025 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.025 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "client.bootstrap-rgw", 2020-11-20T13:56:28.026 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQDSyrdfrZnkCxAAu93JodM4Q1Hm6haxCMGnkw==", 2020-11-20T13:56:28.026 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.026 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile bootstrap-rgw" 2020-11-20T13:56:28.026 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.027 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.027 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "mgr.x", 2020-11-20T13:56:28.027 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfqaJ8HRAAvESNIwCSNYWQq2xSySnCtQ==", 2020-11-20T13:56:28.027 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.027 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow *", 2020-11-20T13:56:28.028 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile mgr", 2020-11-20T13:56:28.028 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:28.028 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:28.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:28.029 INFO:teuthology.orchestra.run.smithi117.stdout: "entity": "mgr.y", 2020-11-20T13:56:28.029 INFO:teuthology.orchestra.run.smithi117.stdout: "key": "AQCtyrdfo6hOGRAAcImFrplXnFTTA0YGGvmJ8g==", 2020-11-20T13:56:28.029 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": { 2020-11-20T13:56:28.029 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": "allow *", 2020-11-20T13:56:28.029 INFO:teuthology.orchestra.run.smithi117.stdout: "mon": "allow profile mgr", 2020-11-20T13:56:28.030 INFO:teuthology.orchestra.run.smithi117.stdout: "osd": "allow *" 2020-11-20T13:56:28.030 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.030 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:28.030 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2020-11-20T13:56:28.030 INFO:teuthology.orchestra.run.smithi117.stdout:} 2020-11-20T13:56:28.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ffbc4051210 msgr2=0x7ffbc40536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ffbc4051210 0x7ffbc40536d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ffbd800b000 tx=0x7ffbd8009040).stop 2020-11-20T13:56:28.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe40ae8b0 msgr2=0x7ffbe40634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe40ae8b0 0x7ffbe40634b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7ffbe409b040 tx=0x7ffbd4007b90).stop 2020-11-20T13:56:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 shutdown_connections 2020-11-20T13:56:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ffbc4051210 0x7ffbc40536d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ffbe414c2b0 0x7ffbe4064820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbe40ae8b0 0x7ffbe40634b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 --2- 172.21.15.117:0/2711326896 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ffbe40b5860 0x7ffbe40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 >> 172.21.15.117:0/2711326896 conn(0x7ffbe405b9f0 msgr2=0x7ffbe409e940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 shutdown_connections 2020-11-20T13:56:28.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:27.983+0000 7ffbeb15d700 1 -- 172.21.15.117:0/2711326896 wait complete. 2020-11-20T13:56:28.035 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mds 2020-11-20T13:56:28.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.141+0000 7f461042f700 1 Processor -- start 2020-11-20T13:56:28.142 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.141+0000 7f461042f700 1 -- start start 2020-11-20T13:56:28.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.142+0000 7f461042f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 0x7f4608143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.142+0000 7f461042f700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460809b920 con 0x7f46080ae6f0 2020-11-20T13:56:28.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.142+0000 7f461042f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080b5510 con 0x7f4608098e30 2020-11-20T13:56:28.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.142+0000 7f461042f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4608144230 con 0x7f46081438e0 2020-11-20T13:56:28.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.142+0000 7f460e1cb700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4608098e30 0x7f460809b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60624/0 (socket says 172.21.15.117:60624) 2020-11-20T13:56:28.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.142+0000 7f460e1cb700 1 -- 172.21.15.117:0/3504050695 learned_addr learned my addr 172.21.15.117:0/3504050695 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:28.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d9ca700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 0x7f4608143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020062584 0 0) 0x7f46080b5510 con 0x7f4608098e30 2020-11-20T13:56:28.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f0003230 con 0x7f4608098e30 2020-11-20T13:56:28.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1394193877 0 0) 0x7f45f0003230 con 0x7f4608098e30 2020-11-20T13:56:28.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 msgr2=0x7f4608143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 0x7f4608143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 >> v1:172.21.15.85:6789/0 conn(0x7f46080ae6f0 legacy=0x7f46080aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080b57b0 con 0x7f4608098e30 2020-11-20T13:56:28.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f45f8001710 con 0x7f4608098e30 2020-11-20T13:56:28.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 >> v1:172.21.15.117:6789/0 conn(0x7f4608098e30 legacy=0x7f460809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 0x7f45f0003720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45f0003c30 0x7f45f00044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f00049c0 0x7f45f0004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608144230 con 0x7f45f0003c30 2020-11-20T13:56:28.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f00041c0 con 0x7f45f00049c0 2020-11-20T13:56:28.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f000b720 con 0x7f46081438e0 2020-11-20T13:56:28.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460e1cb700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 0x7f45f0003720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460d9ca700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45f0003c30 0x7f45f00044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460e9cc700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f00049c0 0x7f45f0004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460e1cb700 1 -- 172.21.15.117:0/3504050695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f00049c0 msgr2=0x7f45f0004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460e1cb700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f00049c0 0x7f45f0004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460e1cb700 1 -- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45f0003c30 msgr2=0x7f45f00044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460e1cb700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45f0003c30 0x7f45f00044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.143+0000 7f460e1cb700 1 -- 172.21.15.117:0/3504050695 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0003230 con 0x7f46081438e0 2020-11-20T13:56:28.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f460e1cb700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 0x7f45f0003720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f45f8001420 tx=0x7f45f80076c0).ready entity=mon.2 client_cookie=484fee174df1e86b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45f80070d0 con 0x7f46081438e0 2020-11-20T13:56:28.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8007230 con 0x7f46081438e0 2020-11-20T13:56:28.153 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f460d1c9700 1 -- 172.21.15.117:0/3504050695 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45f80055e0 con 0x7f46081438e0 2020-11-20T13:56:28.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f461042f700 1 -- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 msgr2=0x7f45f0003720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f461042f700 1 --2- 172.21.15.117:0/3504050695 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f46081438e0 0x7f45f0003720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f45f8001420 tx=0x7f45f80076c0).stop 2020-11-20T13:56:28.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f460e9cc700 1 -- 172.21.15.117:0/3504050695 reap_dead start 2020-11-20T13:56:28.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f461042f700 1 -- 172.21.15.117:0/3504050695 shutdown_connections 2020-11-20T13:56:28.154 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f461042f700 1 -- 172.21.15.117:0/3504050695 >> 172.21.15.117:0/3504050695 conn(0x7f460805b9f0 msgr2=0x7f460805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f461042f700 1 -- 172.21.15.117:0/3504050695 shutdown_connections 2020-11-20T13:56:28.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.144+0000 7f461042f700 1 -- 172.21.15.117:0/3504050695 wait complete. 2020-11-20T13:56:28.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.145+0000 7f461042f700 1 Processor -- start 2020-11-20T13:56:28.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.145+0000 7f461042f700 1 -- start start 2020-11-20T13:56:28.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f461042f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f460809f2d0 0x7f46080b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f461042f700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f460809c7a0 0x7f4608062790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f461042f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4608134ac0 0x7f4608134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f461042f700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46081360f0 con 0x7f460809c7a0 2020-11-20T13:56:28.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f461042f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608063300 con 0x7f4608134ac0 2020-11-20T13:56:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f461042f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4608062dc0 con 0x7f460809f2d0 2020-11-20T13:56:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4608134ac0 0x7f4608134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4608134ac0 0x7f4608134f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41478/0 (socket says 172.21.15.117:41478) 2020-11-20T13:56:28.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 -- 172.21.15.117:0/2984281846 learned_addr learned my addr 172.21.15.117:0/2984281846 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460d9ca700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f460809c7a0 0x7f4608062790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e1cb700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f460809f2d0 0x7f46080b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 -- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f460809f2d0 msgr2=0x7f46080b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f460809f2d0 0x7f46080b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 -- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f460809c7a0 msgr2=0x7f4608062790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f460809c7a0 0x7f4608062790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.146+0000 7f460e9cc700 1 -- 172.21.15.117:0/2984281846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608063060 con 0x7f4608134ac0 2020-11-20T13:56:28.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.147+0000 7f460e9cc700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4608134ac0 0x7f4608134f70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f4608099490 tx=0x7f4600007f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.147+0000 7f45f77fe700 1 -- 172.21.15.117:0/2984281846 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f460000c860 con 0x7f4608134ac0 2020-11-20T13:56:28.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.147+0000 7f45f77fe700 1 -- 172.21.15.117:0/2984281846 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460000c9c0 con 0x7f4608134ac0 2020-11-20T13:56:28.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.147+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460809f6b0 con 0x7f4608134ac0 2020-11-20T13:56:28.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.147+0000 7f45f77fe700 1 -- 172.21.15.117:0/2984281846 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f460000cd20 con 0x7f4608134ac0 2020-11-20T13:56:28.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.147+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46080b03a0 con 0x7f4608134ac0 2020-11-20T13:56:28.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.148+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4608057050 con 0x7f4608134ac0 2020-11-20T13:56:28.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.148+0000 7f45f77fe700 1 -- 172.21.15.117:0/2984281846 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4600014430 con 0x7f4608134ac0 2020-11-20T13:56:28.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.148+0000 7f45f77fe700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f45e8051290 0x7f45e8053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.148+0000 7f45f77fe700 1 -- 172.21.15.117:0/2984281846 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f4600068a40 con 0x7f4608134ac0 2020-11-20T13:56:28.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.148+0000 7f460e1cb700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f45e8051290 0x7f45e8053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.149+0000 7f460e1cb700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f45e8051290 0x7f45e8053750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f45f8019000 tx=0x7f45f8015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.152+0000 7f45f77fe700 1 -- 172.21.15.117:0/2984281846 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f4600040e30 con 0x7f4608134ac0 2020-11-20T13:56:28.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.357+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f460805ca00 con 0x7f4608134ac0 2020-11-20T13:56:28.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.361+0000 7f45f77fe700 1 -- 172.21.15.117:0/2984281846 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4600044450 con 0x7f4608134ac0 2020-11-20T13:56:28.364 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f45e8051290 msgr2=0x7f45e8053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f45e8051290 0x7f45e8053750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f45f8019000 tx=0x7f45f8015000).stop 2020-11-20T13:56:28.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4608134ac0 msgr2=0x7f4608134f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4608134ac0 0x7f4608134f70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f4608099490 tx=0x7f4600007f90).stop 2020-11-20T13:56:28.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 shutdown_connections 2020-11-20T13:56:28.365 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f45e8051290 0x7f45e8053750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f460809f2d0 0x7f46080b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4608134ac0 0x7f4608134f70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 --2- 172.21.15.117:0/2984281846 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f460809c7a0 0x7f4608062790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 >> 172.21.15.117:0/2984281846 conn(0x7f460805b9f0 msgr2=0x7f46080aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.366 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 shutdown_connections 2020-11-20T13:56:28.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.363+0000 7f461042f700 1 -- 172.21.15.117:0/2984281846 wait complete. 2020-11-20T13:56:28.367 INFO:teuthology.orchestra.run.smithi117.stderr:updated 2020-11-20T13:56:28.375 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-mgr 2020-11-20T13:56:28.487 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.485+0000 7f22d47f1700 1 Processor -- start 2020-11-20T13:56:28.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.485+0000 7f22d47f1700 1 -- start start 2020-11-20T13:56:28.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d47f1700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc0be620 0x7f22cc146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d47f1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22cc067b40 0x7f22cc1471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.488 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d47f1700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22cc0b4a00 con 0x7f22cc0b9d90 2020-11-20T13:56:28.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d47f1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc1476a0 con 0x7f22cc067b40 2020-11-20T13:56:28.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d47f1700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc14af20 con 0x7f22cc0be620 2020-11-20T13:56:28.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22cc067b40 0x7f22cc1471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22cc067b40 0x7f22cc1471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41484/0 (socket says 172.21.15.117:41484) 2020-11-20T13:56:28.489 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 -- 172.21.15.117:0/3715746290 learned_addr learned my addr 172.21.15.117:0/3715746290 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:28.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d258d700 1 --2- 172.21.15.117:0/3715746290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc0be620 0x7f22cc146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 -- 172.21.15.117:0/3715746290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc0be620 msgr2=0x7f22cc146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 --2- 172.21.15.117:0/3715746290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc0be620 0x7f22cc146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 -- 172.21.15.117:0/3715746290 >> v1:172.21.15.85:6789/0 conn(0x7f22cc0b9d90 legacy=0x7f22cc0b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.490 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 -- 172.21.15.117:0/3715746290 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc14b1c0 con 0x7f22cc067b40 2020-11-20T13:56:28.491 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.486+0000 7f22d1d8c700 1 --2- 172.21.15.117:0/3715746290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc067b40 0x7f22cc1471b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f22cc14b7e0 tx=0x7f22c8006e80).ready entity=mon.1 client_cookie=9d3d2e9d428c7edc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d158b700 1 -- 172.21.15.117:0/3715746290 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22c80097b0 con 0x7f22cc067b40 2020-11-20T13:56:28.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d158b700 1 -- 172.21.15.117:0/3715746290 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8009910 con 0x7f22cc067b40 2020-11-20T13:56:28.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d158b700 1 -- 172.21.15.117:0/3715746290 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22c8009ac0 con 0x7f22cc067b40 2020-11-20T13:56:28.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 -- 172.21.15.117:0/3715746290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc067b40 msgr2=0x7f22cc1471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.494 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 --2- 172.21.15.117:0/3715746290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc067b40 0x7f22cc1471b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f22cc14b7e0 tx=0x7f22c8006e80).stop 2020-11-20T13:56:28.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 -- 172.21.15.117:0/3715746290 shutdown_connections 2020-11-20T13:56:28.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 --2- 172.21.15.117:0/3715746290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc0be620 0x7f22cc146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 --2- 172.21.15.117:0/3715746290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc067b40 0x7f22cc1471b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 -- 172.21.15.117:0/3715746290 >> 172.21.15.117:0/3715746290 conn(0x7f22cc05c4c0 msgr2=0x7f22cc0bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.495 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 -- 172.21.15.117:0/3715746290 shutdown_connections 2020-11-20T13:56:28.496 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.487+0000 7f22d47f1700 1 -- 172.21.15.117:0/3715746290 wait complete. 2020-11-20T13:56:28.498 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.488+0000 7f22d47f1700 1 Processor -- start 2020-11-20T13:56:28.499 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.488+0000 7f22d47f1700 1 -- start start 2020-11-20T13:56:28.499 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc067b40 0x7f22cc0a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.499 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f22cc0b9d90 0x7f22cc156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.499 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0be620 0x7f22cc156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.499 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0b5af0 con 0x7f22cc0b9d90 2020-11-20T13:56:28.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0b5110 con 0x7f22cc0be620 2020-11-20T13:56:28.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0b53b0 con 0x7f22cc067b40 2020-11-20T13:56:28.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0be620 0x7f22cc156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0be620 0x7f22cc156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41492/0 (socket says 172.21.15.117:41492) 2020-11-20T13:56:28.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 -- 172.21.15.117:0/775697112 learned_addr learned my addr 172.21.15.117:0/775697112 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:28.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d258d700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc067b40 0x7f22cc0a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d1d8c700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f22cc0b9d90 0x7f22cc156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 -- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc067b40 msgr2=0x7f22cc0a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc067b40 0x7f22cc0a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 -- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f22cc0b9d90 msgr2=0x7f22cc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f22cc0b9d90 0x7f22cc156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 -- 172.21.15.117:0/775697112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc0b5650 con 0x7f22cc0be620 2020-11-20T13:56:28.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d2d8e700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0be620 0x7f22cc156e30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f22cc0b5c30 tx=0x7f22c4007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22bb7fe700 1 -- 172.21.15.117:0/775697112 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22c400a560 con 0x7f22cc0be620 2020-11-20T13:56:28.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22bb7fe700 1 -- 172.21.15.117:0/775697112 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c400a6c0 con 0x7f22cc0be620 2020-11-20T13:56:28.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc157320 con 0x7f22cc0be620 2020-11-20T13:56:28.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22bb7fe700 1 -- 172.21.15.117:0/775697112 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22c400aa20 con 0x7f22cc0be620 2020-11-20T13:56:28.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.489+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc08fe40 con 0x7f22cc0be620 2020-11-20T13:56:28.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.491+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc0b57e0 con 0x7f22cc0be620 2020-11-20T13:56:28.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.491+0000 7f22bb7fe700 1 -- 172.21.15.117:0/775697112 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f22c4019030 con 0x7f22cc0be620 2020-11-20T13:56:28.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.491+0000 7f22bb7fe700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f22ac051320 0x7f22ac0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.491+0000 7f22bb7fe700 1 -- 172.21.15.117:0/775697112 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f22c40686d0 con 0x7f22cc0be620 2020-11-20T13:56:28.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.491+0000 7f22d258d700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f22ac051320 0x7f22ac0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.491+0000 7f22d258d700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f22ac051320 0x7f22ac0537e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f22bc00c000 tx=0x7f22bc008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.494+0000 7f22bb7fe700 1 -- 172.21.15.117:0/775697112 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f22c4040b90 con 0x7f22cc0be620 2020-11-20T13:56:28.701 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.701+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f22cc0b4f10 con 0x7f22cc0be620 2020-11-20T13:56:28.705 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.705+0000 7f22bb7fe700 1 -- 172.21.15.117:0/775697112 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7f22c40441b0 con 0x7f22cc0be620 2020-11-20T13:56:28.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f22ac051320 msgr2=0x7f22ac0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f22ac051320 0x7f22ac0537e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f22bc00c000 tx=0x7f22bc008040).stop 2020-11-20T13:56:28.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0be620 msgr2=0x7f22cc156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0be620 0x7f22cc156e30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f22cc0b5c30 tx=0x7f22c4007ef0).stop 2020-11-20T13:56:28.707 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 shutdown_connections 2020-11-20T13:56:28.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f22ac051320 0x7f22ac0537e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f22cc067b40 0x7f22cc0a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0be620 0x7f22cc156e30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.708 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 --2- 172.21.15.117:0/775697112 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f22cc0b9d90 0x7f22cc156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.709 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 >> 172.21.15.117:0/775697112 conn(0x7f22cc05c4c0 msgr2=0x7f22cc0bba90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.709 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 shutdown_connections 2020-11-20T13:56:28.709 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.706+0000 7f22d47f1700 1 -- 172.21.15.117:0/775697112 wait complete. 2020-11-20T13:56:28.709 INFO:teuthology.orchestra.run.smithi117.stderr:updated 2020-11-20T13:56:28.717 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-osd 2020-11-20T13:56:28.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.823+0000 7fc8b8cbc700 1 Processor -- start 2020-11-20T13:56:28.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.823+0000 7fc8b8cbc700 1 -- start start 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b8cbc700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b4098e30 0x7fc8b409b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b8cbc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc8b409b700 0x7fc8b41438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b8cbc700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fc8b4143dd0 0x7fc8b40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b8cbc700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40b5510 con 0x7fc8b4143dd0 2020-11-20T13:56:28.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b8cbc700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40b57b0 con 0x7fc8b409b700 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b8cbc700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40b5a50 con 0x7fc8b4098e30 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc8b409b700 0x7fc8b41438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc8b409b700 0x7fc8b41438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41498/0 (socket says 172.21.15.117:41498) 2020-11-20T13:56:28.827 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 -- 172.21.15.117:0/2452949698 learned_addr learned my addr 172.21.15.117:0/2452949698 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:28.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b2d9d700 1 --2- 172.21.15.117:0/2452949698 >> v2:172.21.15.85:3300/0 conn(0x7fc8b4143dd0 0x7fc8b40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b259c700 1 --2- 172.21.15.117:0/2452949698 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b4098e30 0x7fc8b409b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 -- 172.21.15.117:0/2452949698 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b4098e30 msgr2=0x7fc8b409b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 --2- 172.21.15.117:0/2452949698 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b4098e30 0x7fc8b409b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.828 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 -- 172.21.15.117:0/2452949698 >> v2:172.21.15.85:3300/0 conn(0x7fc8b4143dd0 msgr2=0x7fc8b40b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:28.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 --2- 172.21.15.117:0/2452949698 >> v2:172.21.15.85:3300/0 conn(0x7fc8b4143dd0 0x7fc8b40b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 -- 172.21.15.117:0/2452949698 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4099210 con 0x7fc8b409b700 2020-11-20T13:56:28.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b259c700 1 --2- 172.21.15.117:0/2452949698 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b4098e30 0x7fc8b409b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:28.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b1d9b700 1 --2- 172.21.15.117:0/2452949698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b409b700 0x7fc8b41438e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fc89c0032b0 tx=0x7fc89c009270).ready entity=mon.1 client_cookie=14cb9473afd09c84 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.829 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b159a700 1 -- 172.21.15.117:0/2452949698 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc89c00bc00 con 0x7fc8b409b700 2020-11-20T13:56:28.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.824+0000 7fc8b159a700 1 -- 172.21.15.117:0/2452949698 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c00bd60 con 0x7fc8b409b700 2020-11-20T13:56:28.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b159a700 1 -- 172.21.15.117:0/2452949698 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc89c011400 con 0x7fc8b409b700 2020-11-20T13:56:28.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2452949698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b409b700 msgr2=0x7fc8b41438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2452949698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b409b700 0x7fc8b41438e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fc89c0032b0 tx=0x7fc89c009270).stop 2020-11-20T13:56:28.830 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2452949698 shutdown_connections 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2452949698 >> v2:172.21.15.85:3300/0 conn(0x7fc8b4143dd0 0x7fc8b40b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2452949698 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b4098e30 0x7fc8b409b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2452949698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b409b700 0x7fc8b41438e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2452949698 >> 172.21.15.117:0/2452949698 conn(0x7fc8b405b9f0 msgr2=0x7fc8b405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2452949698 shutdown_connections 2020-11-20T13:56:28.831 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.825+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2452949698 wait complete. 2020-11-20T13:56:28.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.826+0000 7fc8b8cbc700 1 Processor -- start 2020-11-20T13:56:28.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.826+0000 7fc8b8cbc700 1 -- start start 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b4098e30 0x7fc8b41445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b409b700 0x7fc8b41468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc8b4143dd0 0x7fc8b40b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b413f4a0 con 0x7fc8b4143dd0 2020-11-20T13:56:28.835 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4157420 con 0x7fc8b4098e30 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40b0840 con 0x7fc8b409b700 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b4098e30 0x7fc8b41445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b4098e30 0x7fc8b41445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41502/0 (socket says 172.21.15.117:41502) 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 -- 172.21.15.117:0/2951262746 learned_addr learned my addr 172.21.15.117:0/2951262746 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:28.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b2d9d700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc8b4143dd0 0x7fc8b40b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b1d9b700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b409b700 0x7fc8b41468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 -- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b409b700 msgr2=0x7fc8b41468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b409b700 0x7fc8b41468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 -- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc8b4143dd0 msgr2=0x7fc8b40b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:28.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc8b4143dd0 0x7fc8b40b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:28.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 -- 172.21.15.117:0/2951262746 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40b0ae0 con 0x7fc8b4098e30 2020-11-20T13:56:28.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b259c700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b4098e30 0x7fc8b41445e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fc8a4007000 tx=0x7fc8a4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8ab7fe700 1 -- 172.21.15.117:0/2951262746 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8a4010580 con 0x7fc8b4098e30 2020-11-20T13:56:28.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8ab7fe700 1 -- 172.21.15.117:0/2951262746 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a40106e0 con 0x7fc8b4098e30 2020-11-20T13:56:28.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4149270 con 0x7fc8b4098e30 2020-11-20T13:56:28.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8ab7fe700 1 -- 172.21.15.117:0/2951262746 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8a4010a40 con 0x7fc8b4098e30 2020-11-20T13:56:28.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.827+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b4157160 con 0x7fc8b4098e30 2020-11-20T13:56:28.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.828+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b4057050 con 0x7fc8b4098e30 2020-11-20T13:56:28.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.828+0000 7fc8ab7fe700 1 -- 172.21.15.117:0/2951262746 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fc8a4010ba0 con 0x7fc8b4098e30 2020-11-20T13:56:28.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.829+0000 7fc8ab7fe700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc88c051290 0x7fc88c053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:28.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.829+0000 7fc8ab7fe700 1 -- 172.21.15.117:0/2951262746 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fc8a4068ff0 con 0x7fc8b4098e30 2020-11-20T13:56:28.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.829+0000 7fc8b1d9b700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc88c051290 0x7fc88c053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:28.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.829+0000 7fc8b1d9b700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc88c051290 0x7fc88c053750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc89c018000 tx=0x7fc89c016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:28.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:28.832+0000 7fc8ab7fe700 1 -- 172.21.15.117:0/2951262746 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fc8a4041450 con 0x7fc8b4098e30 2020-11-20T13:56:29.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.035+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fc8b405ca00 con 0x7fc8b4098e30 2020-11-20T13:56:29.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.039+0000 7fc8ab7fe700 1 -- 172.21.15.117:0/2951262746 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc8a4044a70 con 0x7fc8b4098e30 2020-11-20T13:56:29.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc88c051290 msgr2=0x7fc88c053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc88c051290 0x7fc88c053750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc89c018000 tx=0x7fc89c016000).stop 2020-11-20T13:56:29.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b4098e30 msgr2=0x7fc8b41445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b4098e30 0x7fc8b41445e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fc8a4007000 tx=0x7fc8a4008040).stop 2020-11-20T13:56:29.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 shutdown_connections 2020-11-20T13:56:29.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fc88c051290 0x7fc88c053750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc8b4143dd0 0x7fc8b40b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8b4098e30 0x7fc8b41445e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 --2- 172.21.15.117:0/2951262746 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc8b409b700 0x7fc8b41468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 >> 172.21.15.117:0/2951262746 conn(0x7fc8b405b9f0 msgr2=0x7fc8b40b6780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.040+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 shutdown_connections 2020-11-20T13:56:29.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.041+0000 7fc8b8cbc700 1 -- 172.21.15.117:0/2951262746 wait complete. 2020-11-20T13:56:29.044 INFO:teuthology.orchestra.run.smithi117.stderr:updated 2020-11-20T13:56:29.051 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd 2020-11-20T13:56:29.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.157+0000 7fe6b6bcf700 1 Processor -- start 2020-11-20T13:56:29.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.157+0000 7fe6b6bcf700 1 -- start start 2020-11-20T13:56:29.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b6bcf700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6b00ae8b0 0x7fe6b00aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b6bcf700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b00b5860 0x7fe6b0098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b6bcf700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b00b4e80 con 0x7fe6b0099080 2020-11-20T13:56:29.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b6bcf700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b00994f0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b6bcf700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00b5120 con 0x7fe6b00b5860 2020-11-20T13:56:29.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b496b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6b00ae8b0 0x7fe6b00aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b496b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6b00ae8b0 0x7fe6b00aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41514/0 (socket says 172.21.15.117:41514) 2020-11-20T13:56:29.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6b496b700 1 -- 172.21.15.117:0/1372369738 learned_addr learned my addr 172.21.15.117:0/1372369738 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:29.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.158+0000 7fe6affff700 1 --2- 172.21.15.117:0/1372369738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b00b5860 0x7fe6b0098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b496b700 1 -- 172.21.15.117:0/1372369738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b00b5860 msgr2=0x7fe6b0098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:29.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b496b700 1 --2- 172.21.15.117:0/1372369738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b00b5860 0x7fe6b0098b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b496b700 1 -- 172.21.15.117:0/1372369738 >> v1:172.21.15.85:6789/0 conn(0x7fe6b0099080 legacy=0x7fe6b0146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b496b700 1 -- 172.21.15.117:0/1372369738 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b00b53c0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b496b700 1 --2- 172.21.15.117:0/1372369738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b00aec90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fe6a4002f30 tx=0x7fe6a4006e80).ready entity=mon.1 client_cookie=59397cad356c34c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6af7fe700 1 -- 172.21.15.117:0/1372369738 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6a40097f0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6af7fe700 1 -- 172.21.15.117:0/1372369738 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a4009950 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6af7fe700 1 -- 172.21.15.117:0/1372369738 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6a4009cb0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1372369738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 msgr2=0x7fe6b00aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1372369738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b00aec90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fe6a4002f30 tx=0x7fe6a4006e80).stop 2020-11-20T13:56:29.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1372369738 shutdown_connections 2020-11-20T13:56:29.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1372369738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b00aec90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.159+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1372369738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b00b5860 0x7fe6b0098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.160+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1372369738 >> 172.21.15.117:0/1372369738 conn(0x7fe6b005b9f0 msgr2=0x7fe6b005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.160+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1372369738 shutdown_connections 2020-11-20T13:56:29.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.160+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1372369738 wait complete. 2020-11-20T13:56:29.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.160+0000 7fe6b6bcf700 1 Processor -- start 2020-11-20T13:56:29.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b6bcf700 1 -- start start 2020-11-20T13:56:29.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b6bcf700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b0099080 0x7fe6b01574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b6bcf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b009f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b6bcf700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe6b00b5860 0x7fe6b00b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b6bcf700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00b18f0 con 0x7fe6b00b5860 2020-11-20T13:56:29.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b6bcf700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00b1b90 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b6bcf700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00b2520 con 0x7fe6b0099080 2020-11-20T13:56:29.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b009f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b009f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41518/0 (socket says 172.21.15.117:41518) 2020-11-20T13:56:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6affff700 1 -- 172.21.15.117:0/1590307838 learned_addr learned my addr 172.21.15.117:0/1590307838 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.161+0000 7fe6b496b700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b0099080 0x7fe6b01574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6b516c700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe6b00b5860 0x7fe6b00b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6affff700 1 -- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b0099080 msgr2=0x7fe6b01574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6affff700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b0099080 0x7fe6b01574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6affff700 1 -- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe6b00b5860 msgr2=0x7fe6b00b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6affff700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe6b00b5860 0x7fe6b00b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6affff700 1 -- 172.21.15.117:0/1590307838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b00b26e0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6affff700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b009f040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fe69c00c000 tx=0x7fe69c006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6adffb700 1 -- 172.21.15.117:0/1590307838 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c00ca10 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.171 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6adffb700 1 -- 172.21.15.117:0/1590307838 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00cb70 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b00b3260 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6adffb700 1 -- 172.21.15.117:0/1590307838 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe69c0123f0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.162+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b0062ae0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.163+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0057050 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.163+0000 7fe6adffb700 1 -- 172.21.15.117:0/1590307838 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fe69c012550 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.163+0000 7fe6adffb700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe698051210 0x7fe6980536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.163+0000 7fe6adffb700 1 -- 172.21.15.117:0/1590307838 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fe69c0690a0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.163+0000 7fe6b496b700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe698051210 0x7fe6980536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.164+0000 7fe6b496b700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe698051210 0x7fe6980536d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fe6a401c000 tx=0x7fe6a4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.167+0000 7fe6adffb700 1 -- 172.21.15.117:0/1590307838 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fe69c0415a0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.376 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.374+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fe6b00b5fd0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.378+0000 7fe6adffb700 1 -- 172.21.15.117:0/1590307838 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe69c044bc0 con 0x7fe6b00ae8b0 2020-11-20T13:56:29.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.379+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe698051210 msgr2=0x7fe6980536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.379+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe698051210 0x7fe6980536d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fe6a401c000 tx=0x7fe6a4015000).stop 2020-11-20T13:56:29.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.379+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 msgr2=0x7fe6b009f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.379+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b009f040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fe69c00c000 tx=0x7fe69c006dc0).stop 2020-11-20T13:56:29.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 shutdown_connections 2020-11-20T13:56:29.381 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fe698051210 0x7fe6980536d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe6b0099080 0x7fe6b01574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe6b00b5860 0x7fe6b00b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 --2- 172.21.15.117:0/1590307838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b00ae8b0 0x7fe6b009f040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 >> 172.21.15.117:0/1590307838 conn(0x7fe6b005b9f0 msgr2=0x7fe6b009bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 shutdown_connections 2020-11-20T13:56:29.383 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.380+0000 7fe6b6bcf700 1 -- 172.21.15.117:0/1590307838 wait complete. 2020-11-20T13:56:29.383 INFO:teuthology.orchestra.run.smithi117.stderr:updated 2020-11-20T13:56:29.390 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rbd-mirror 2020-11-20T13:56:29.497 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.496+0000 7f31d56e4700 1 Processor -- start 2020-11-20T13:56:29.497 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.496+0000 7f31d56e4700 1 -- start start 2020-11-20T13:56:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.497+0000 7f31d56e4700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f31d0143d40 0x7f31d00aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.499 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.497+0000 7f31d56e4700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d009b920 con 0x7f31d0098e30 2020-11-20T13:56:29.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.497+0000 7f31d56e4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d01441b0 con 0x7f31d01438e0 2020-11-20T13:56:29.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.497+0000 7f31d56e4700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d00b5510 con 0x7f31d0143d40 2020-11-20T13:56:29.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.497+0000 7f31ce7fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f31d01438e0 0x7f31d00ae6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60684/0 (socket says 172.21.15.117:60684) 2020-11-20T13:56:29.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.497+0000 7f31ce7fc700 1 -- 172.21.15.117:0/2238852537 learned_addr learned my addr 172.21.15.117:0/2238852537 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:29.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cf7fe700 1 --2- 172.21.15.117:0/2238852537 >> v2:172.21.15.85:3300/0 conn(0x7f31d0143d40 0x7f31d00aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cdffb700 1 -- 172.21.15.117:0/2238852537 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735592361 0 0) 0x7f31d01441b0 con 0x7f31d01438e0 2020-11-20T13:56:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cdffb700 1 -- 172.21.15.117:0/2238852537 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b4003230 con 0x7f31d01438e0 2020-11-20T13:56:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cf7fe700 1 -- 172.21.15.117:0/2238852537 >> v1:172.21.15.117:6789/0 conn(0x7f31d01438e0 legacy=0x7f31d00ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cf7fe700 1 -- 172.21.15.117:0/2238852537 >> v1:172.21.15.85:6789/0 conn(0x7f31d0098e30 legacy=0x7f31d009b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cdffb700 1 -- 172.21.15.117:0/2238852537 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554931655 0 0) 0x7f31d009b920 con 0x7f31d0098e30 2020-11-20T13:56:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cf7fe700 1 -- 172.21.15.117:0/2238852537 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d00b57b0 con 0x7f31d0143d40 2020-11-20T13:56:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cf7fe700 1 --2- 172.21.15.117:0/2238852537 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 0x7f31d00aee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f31c80032b0 tx=0x7f31c80091a0).ready entity=mon.0 client_cookie=85b302efc041bc6a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cdffb700 1 -- 172.21.15.117:0/2238852537 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c800b9a0 con 0x7f31d0143d40 2020-11-20T13:56:29.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cdffb700 1 -- 172.21.15.117:0/2238852537 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c800bb00 con 0x7f31d0143d40 2020-11-20T13:56:29.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31cdffb700 1 -- 172.21.15.117:0/2238852537 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c800bc70 con 0x7f31d0143d40 2020-11-20T13:56:29.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31d56e4700 1 -- 172.21.15.117:0/2238852537 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 msgr2=0x7f31d00aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.498+0000 7f31d56e4700 1 --2- 172.21.15.117:0/2238852537 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 0x7f31d00aee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f31c80032b0 tx=0x7f31c80091a0).stop 2020-11-20T13:56:29.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.499+0000 7f31d56e4700 1 -- 172.21.15.117:0/2238852537 shutdown_connections 2020-11-20T13:56:29.505 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.499+0000 7f31d56e4700 1 --2- 172.21.15.117:0/2238852537 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 0x7f31d00aee00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.505 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.499+0000 7f31d56e4700 1 -- 172.21.15.117:0/2238852537 >> 172.21.15.117:0/2238852537 conn(0x7f31d005b9f0 msgr2=0x7f31d005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.505 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.499+0000 7f31d56e4700 1 -- 172.21.15.117:0/2238852537 shutdown_connections 2020-11-20T13:56:29.505 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.499+0000 7f31d56e4700 1 -- 172.21.15.117:0/2238852537 wait complete. 2020-11-20T13:56:29.508 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 Processor -- start 2020-11-20T13:56:29.508 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 -- start start 2020-11-20T13:56:29.509 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f31d0098e30 0x7f31d0140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.509 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d01438e0 0x7f31d0145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.509 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 0x7f31d0157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.509 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0067680 con 0x7f31d0143d40 2020-11-20T13:56:29.510 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0067920 con 0x7f31d01438e0 2020-11-20T13:56:29.510 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.500+0000 7f31d56e4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0157780 con 0x7f31d0098e30 2020-11-20T13:56:29.510 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d01438e0 0x7f31d0145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d01438e0 0x7f31d0145620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41528/0 (socket says 172.21.15.117:41528) 2020-11-20T13:56:29.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 -- 172.21.15.117:0/23620881 learned_addr learned my addr 172.21.15.117:0/23620881 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:29.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31cf7fe700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 0x7f31d0157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.511 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ceffd700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f31d0098e30 0x7f31d0140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 -- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f31d0098e30 msgr2=0x7f31d0140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f31d0098e30 0x7f31d0140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 -- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 msgr2=0x7f31d0157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 0x7f31d0157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 -- 172.21.15.117:0/23620881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d00b3290 con 0x7f31d01438e0 2020-11-20T13:56:29.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31ce7fc700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d01438e0 0x7f31d0145620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f31c00010d0 tx=0x7f31c0007a70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31bffff700 1 -- 172.21.15.117:0/23620881 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c00072e0 con 0x7f31d01438e0 2020-11-20T13:56:29.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31bffff700 1 -- 172.21.15.117:0/23620881 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0007440 con 0x7f31d01438e0 2020-11-20T13:56:29.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d00671e0 con 0x7f31d01438e0 2020-11-20T13:56:29.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31bffff700 1 -- 172.21.15.117:0/23620881 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31c00057d0 con 0x7f31d01438e0 2020-11-20T13:56:29.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.501+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d0145d50 con 0x7f31d01438e0 2020-11-20T13:56:29.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.502+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d00674f0 con 0x7f31d01438e0 2020-11-20T13:56:29.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.502+0000 7f31bffff700 1 -- 172.21.15.117:0/23620881 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f31c0010070 con 0x7f31d01438e0 2020-11-20T13:56:29.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.502+0000 7f31bffff700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f31ac051290 0x7f31ac053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.502+0000 7f31bffff700 1 -- 172.21.15.117:0/23620881 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f31c0068420 con 0x7f31d01438e0 2020-11-20T13:56:29.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.503+0000 7f31ceffd700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f31ac051290 0x7f31ac053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.503+0000 7f31ceffd700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f31ac051290 0x7f31ac053750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f31c400b000 tx=0x7f31c4009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.506+0000 7f31bffff700 1 -- 172.21.15.117:0/23620881 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f31c0040a30 con 0x7f31d01438e0 2020-11-20T13:56:29.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.711+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f31d0057050 con 0x7f31d01438e0 2020-11-20T13:56:29.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.714+0000 7f31bffff700 1 -- 172.21.15.117:0/23620881 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f31c0018070 con 0x7f31d01438e0 2020-11-20T13:56:29.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f31ac051290 msgr2=0x7f31ac053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f31ac051290 0x7f31ac053750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f31c400b000 tx=0x7f31c4009040).stop 2020-11-20T13:56:29.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d01438e0 msgr2=0x7f31d0145620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d01438e0 0x7f31d0145620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f31c00010d0 tx=0x7f31c0007a70).stop 2020-11-20T13:56:29.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 shutdown_connections 2020-11-20T13:56:29.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f31ac051290 0x7f31ac053750 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f31d0098e30 0x7f31d0140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f31d0143d40 0x7f31d0157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 --2- 172.21.15.117:0/23620881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d01438e0 0x7f31d0145620 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 >> 172.21.15.117:0/23620881 conn(0x7f31d005b9f0 msgr2=0x7f31d01422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 shutdown_connections 2020-11-20T13:56:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.716+0000 7f31d56e4700 1 -- 172.21.15.117:0/23620881 wait complete. 2020-11-20T13:56:29.720 INFO:teuthology.orchestra.run.smithi117.stderr:updated 2020-11-20T13:56:29.726 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early auth del client.bootstrap-rgw 2020-11-20T13:56:29.833 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.833+0000 7f85b7565700 1 Processor -- start 2020-11-20T13:56:29.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.833+0000 7f85b7565700 1 -- start start 2020-11-20T13:56:29.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b7565700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f85b0098e30 0x7f85b009b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b7565700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00b5510 con 0x7f85b009b700 2020-11-20T13:56:29.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b7565700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00b57b0 con 0x7f85b00ae6f0 2020-11-20T13:56:29.836 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b7565700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b00b5a50 con 0x7f85b0098e30 2020-11-20T13:56:29.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b5301700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f85b0098e30 0x7f85b009b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b5b02700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f85b00ae6f0 0x7f85b00aeb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60696/0 (socket says 172.21.15.117:60696) 2020-11-20T13:56:29.837 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b5301700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f85b0098e30 0x7f85b009b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41536/0 (socket says 172.21.15.117:41536) 2020-11-20T13:56:29.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b5b02700 1 -- 172.21.15.117:0/954264407 learned_addr learned my addr 172.21.15.117:0/954264407 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:29.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b5301700 1 -- 172.21.15.117:0/954264407 >> v1:172.21.15.117:6789/0 conn(0x7f85b00ae6f0 legacy=0x7f85b00aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b5301700 1 -- 172.21.15.117:0/954264407 >> v1:172.21.15.85:6789/0 conn(0x7f85b009b700 legacy=0x7f85b01438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:29.838 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f859ffff700 1 -- 172.21.15.117:0/954264407 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707294980 0 0) 0x7f85b00b57b0 con 0x7f85b00ae6f0 2020-11-20T13:56:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.834+0000 7f85b5301700 1 -- 172.21.15.117:0/954264407 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0099210 con 0x7f85b0098e30 2020-11-20T13:56:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b5301700 1 --2- 172.21.15.117:0/954264407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009b210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f85b0060b70 tx=0x7f85a0006dc0).ready entity=mon.1 client_cookie=b23055bc3ddb01f5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f859ffff700 1 -- 172.21.15.117:0/954264407 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a00097b0 con 0x7f85b0098e30 2020-11-20T13:56:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f859ffff700 1 -- 172.21.15.117:0/954264407 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0009910 con 0x7f85b0098e30 2020-11-20T13:56:29.839 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f859ffff700 1 -- 172.21.15.117:0/954264407 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a0009c70 con 0x7f85b0098e30 2020-11-20T13:56:29.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b7565700 1 -- 172.21.15.117:0/954264407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 msgr2=0x7f85b009b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.840 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b7565700 1 --2- 172.21.15.117:0/954264407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009b210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f85b0060b70 tx=0x7f85a0006dc0).stop 2020-11-20T13:56:29.841 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b7565700 1 -- 172.21.15.117:0/954264407 shutdown_connections 2020-11-20T13:56:29.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b7565700 1 --2- 172.21.15.117:0/954264407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009b210 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b7565700 1 -- 172.21.15.117:0/954264407 >> 172.21.15.117:0/954264407 conn(0x7f85b005b9f0 msgr2=0x7f85b005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:29.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b7565700 1 -- 172.21.15.117:0/954264407 shutdown_connections 2020-11-20T13:56:29.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.835+0000 7f85b7565700 1 -- 172.21.15.117:0/954264407 wait complete. 2020-11-20T13:56:29.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.836+0000 7f85b7565700 1 Processor -- start 2020-11-20T13:56:29.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.836+0000 7f85b7565700 1 -- start start 2020-11-20T13:56:29.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b7565700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b7565700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f85b009b700 0x7f85b00b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b7565700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f85b00ae6f0 0x7f85b013f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b7565700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0145140 con 0x7f85b009b700 2020-11-20T13:56:29.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b7565700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b013f7b0 con 0x7f85b0098e30 2020-11-20T13:56:29.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b7565700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0145370 con 0x7f85b00ae6f0 2020-11-20T13:56:29.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41544/0 (socket says 172.21.15.117:41544) 2020-11-20T13:56:29.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 -- 172.21.15.117:0/4198077140 learned_addr learned my addr 172.21.15.117:0/4198077140 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:29.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5b02700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f85b00ae6f0 0x7f85b013f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b4b00700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f85b009b700 0x7f85b00b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 -- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f85b00ae6f0 msgr2=0x7f85b013f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f85b00ae6f0 0x7f85b013f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 -- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f85b009b700 msgr2=0x7f85b00b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:29.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f85b009b700 0x7f85b00b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:29.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 -- 172.21.15.117:0/4198077140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0145610 con 0x7f85b0098e30 2020-11-20T13:56:29.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f85b5301700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009f580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f85b0060b70 tx=0x7f85a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.837+0000 7f859e7fc700 1 -- 172.21.15.117:0/4198077140 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a0011b80 con 0x7f85b0098e30 2020-11-20T13:56:29.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.838+0000 7f859e7fc700 1 -- 172.21.15.117:0/4198077140 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0011ce0 con 0x7f85b0098e30 2020-11-20T13:56:29.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.838+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b009d920 con 0x7f85b0098e30 2020-11-20T13:56:29.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.838+0000 7f859e7fc700 1 -- 172.21.15.117:0/4198077140 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a001d640 con 0x7f85b0098e30 2020-11-20T13:56:29.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.838+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b009de10 con 0x7f85b0098e30 2020-11-20T13:56:29.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.839+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b0057050 con 0x7f85b0098e30 2020-11-20T13:56:29.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.839+0000 7f859e7fc700 1 -- 172.21.15.117:0/4198077140 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f85a000f070 con 0x7f85b0098e30 2020-11-20T13:56:29.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.839+0000 7f859e7fc700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8590051210 0x7f85900536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:29.853 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.839+0000 7f859e7fc700 1 -- 172.21.15.117:0/4198077140 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f85a006a9a0 con 0x7f85b0098e30 2020-11-20T13:56:29.853 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.839+0000 7f85b4b00700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8590051210 0x7f85900536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:29.853 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.839+0000 7f85b4b00700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8590051210 0x7f85900536d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f85ac00d000 tx=0x7f85ac007b60).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:29.853 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:29.842+0000 7f859e7fc700 1 -- 172.21.15.117:0/4198077140 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f85a0042f00 con 0x7f85b0098e30 2020-11-20T13:56:30.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.048+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f85b005ca00 con 0x7f85b0098e30 2020-11-20T13:56:30.053 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.053+0000 7f859e7fc700 1 -- 172.21.15.117:0/4198077140 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f85a0046520 con 0x7f85b0098e30 2020-11-20T13:56:30.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8590051210 msgr2=0x7f85900536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8590051210 0x7f85900536d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f85ac00d000 tx=0x7f85ac007b60).stop 2020-11-20T13:56:30.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 msgr2=0x7f85b009f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.055 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009f580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f85b0060b70 tx=0x7f85a0015000).stop 2020-11-20T13:56:30.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 shutdown_connections 2020-11-20T13:56:30.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8590051210 0x7f85900536d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f85b00ae6f0 0x7f85b013f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85b0098e30 0x7f85b009f580 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.058 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 --2- 172.21.15.117:0/4198077140 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f85b009b700 0x7f85b00b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.058 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 >> 172.21.15.117:0/4198077140 conn(0x7f85b005b9f0 msgr2=0x7f85b00b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.058 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 shutdown_connections 2020-11-20T13:56:30.058 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.054+0000 7f85b7565700 1 -- 172.21.15.117:0/4198077140 wait complete. 2020-11-20T13:56:30.059 INFO:teuthology.orchestra.run.smithi117.stderr:updated 2020-11-20T13:56:30.065 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:30.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.172+0000 7f3e0ad7b700 1 Processor -- start 2020-11-20T13:56:30.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.172+0000 7f3e0ad7b700 1 -- start start 2020-11-20T13:56:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e0ad7b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e040b0080 0x7f3e04146420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e0ad7b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e04149190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.175 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e0ad7b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e04098be0 con 0x7f3e0409bfe0 2020-11-20T13:56:30.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e0ad7b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e04098e80 con 0x7f3e040b0080 2020-11-20T13:56:30.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e0ad7b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04099120 con 0x7f3e0409c7a0 2020-11-20T13:56:30.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e08b17700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e040b0080 0x7f3e04146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e03fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3e0409bfe0 0x7f3e04148a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60710/0 (socket says 172.21.15.117:60710) 2020-11-20T13:56:30.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e03fff700 1 -- 172.21.15.117:0/1978595600 learned_addr learned my addr 172.21.15.117:0/1978595600 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:30.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e08b17700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e040b0080 0x7f3e04146420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41548/0 (socket says 172.21.15.117:41548) 2020-11-20T13:56:30.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e09318700 1 --2- 172.21.15.117:0/1978595600 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e04149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e08b17700 1 -- 172.21.15.117:0/1978595600 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 msgr2=0x7f3e04149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e08b17700 1 --2- 172.21.15.117:0/1978595600 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e04149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e08b17700 1 -- 172.21.15.117:0/1978595600 >> v1:172.21.15.117:6789/0 conn(0x7f3e0409bfe0 legacy=0x7f3e04148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e037fe700 1 -- 172.21.15.117:0/1978595600 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787896812 0 0) 0x7f3e04098be0 con 0x7f3e0409bfe0 2020-11-20T13:56:30.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e08b17700 1 -- 172.21.15.117:0/1978595600 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040b0460 con 0x7f3e040b0080 2020-11-20T13:56:30.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e09318700 1 --2- 172.21.15.117:0/1978595600 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e04149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:30.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e08b17700 1 --2- 172.21.15.117:0/1978595600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e04146420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3dfc0032b0 tx=0x7f3dfc0091a0).ready entity=mon.1 client_cookie=3ebf593e0dfbccf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e037fe700 1 -- 172.21.15.117:0/1978595600 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dfc00ba50 con 0x7f3e040b0080 2020-11-20T13:56:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e037fe700 1 -- 172.21.15.117:0/1978595600 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc00bbb0 con 0x7f3e040b0080 2020-11-20T13:56:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e037fe700 1 -- 172.21.15.117:0/1978595600 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dfc00bd20 con 0x7f3e040b0080 2020-11-20T13:56:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/1978595600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 msgr2=0x7f3e04146420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.173+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/1978595600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e04146420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3dfc0032b0 tx=0x7f3dfc0091a0).stop 2020-11-20T13:56:30.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.174+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/1978595600 shutdown_connections 2020-11-20T13:56:30.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.174+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/1978595600 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e04149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.174+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/1978595600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e04146420 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.174+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/1978595600 >> 172.21.15.117:0/1978595600 conn(0x7f3e0405b9f0 msgr2=0x7f3e0405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.174+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/1978595600 shutdown_connections 2020-11-20T13:56:30.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.174+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/1978595600 wait complete. 2020-11-20T13:56:30.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.175+0000 7f3e0ad7b700 1 Processor -- start 2020-11-20T13:56:30.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.175+0000 7f3e0ad7b700 1 -- start start 2020-11-20T13:56:30.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3e0409bfe0 0x7f3e040b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e0413fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e0413fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04157a00 con 0x7f3e0409bfe0 2020-11-20T13:56:30.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04157ca0 con 0x7f3e040b0080 2020-11-20T13:56:30.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04157f40 con 0x7f3e0409c7a0 2020-11-20T13:56:30.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e0413fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e0413fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41558/0 (socket says 172.21.15.117:41558) 2020-11-20T13:56:30.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 -- 172.21.15.117:0/945324533 learned_addr learned my addr 172.21.15.117:0/945324533 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:30.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e03fff700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e0413fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e08b17700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3e0409bfe0 0x7f3e040b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 -- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 msgr2=0x7f3e0413fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e0413fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 -- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3e0409bfe0 msgr2=0x7f3e040b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3e0409bfe0 0x7f3e040b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 -- 172.21.15.117:0/945324533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e041583e0 con 0x7f3e040b0080 2020-11-20T13:56:30.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e03fff700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e0413fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:30.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e08b17700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3e0409bfe0 0x7f3e040b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:30.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e09318700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e0413fef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f3df4006910 tx=0x7f3df4008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e01ffb700 1 -- 172.21.15.117:0/945324533 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3df4009980 con 0x7f3e040b0080 2020-11-20T13:56:30.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e01ffb700 1 -- 172.21.15.117:0/945324533 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df4009ae0 con 0x7f3e040b0080 2020-11-20T13:56:30.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e01ffb700 1 -- 172.21.15.117:0/945324533 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3df4009e40 con 0x7f3e040b0080 2020-11-20T13:56:30.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e040ba4c0 con 0x7f3e040b0080 2020-11-20T13:56:30.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.176+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e04157720 con 0x7f3e040b0080 2020-11-20T13:56:30.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.177+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04158100 con 0x7f3e040b0080 2020-11-20T13:56:30.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.177+0000 7f3e01ffb700 1 -- 172.21.15.117:0/945324533 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3df4013480 con 0x7f3e040b0080 2020-11-20T13:56:30.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.178+0000 7f3e01ffb700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3de4051320 0x7f3de40537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.178+0000 7f3e01ffb700 1 -- 172.21.15.117:0/945324533 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3df40696b0 con 0x7f3e040b0080 2020-11-20T13:56:30.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.178+0000 7f3e08b17700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3de4051320 0x7f3de40537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.178+0000 7f3e08b17700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3de4051320 0x7f3de40537e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f3dfc017000 tx=0x7f3dfc016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.181+0000 7f3e01ffb700 1 -- 172.21.15.117:0/945324533 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3df4041b10 con 0x7f3e040b0080 2020-11-20T13:56:30.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.330+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e04057050 con 0x7f3e040b0080 2020-11-20T13:56:30.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.331+0000 7f3e01ffb700 1 -- 172.21.15.117:0/945324533 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+13462 (secure 0 0 0) 0x7f3df4045130 con 0x7f3e040b0080 2020-11-20T13:56:30.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:30.332 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":31,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:56:27.633705+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:30.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3de4051320 msgr2=0x7f3de40537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3de4051320 0x7f3de40537e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f3dfc017000 tx=0x7f3dfc016000).stop 2020-11-20T13:56:30.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 msgr2=0x7f3e0413fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e0413fef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f3df4006910 tx=0x7f3df4008040).stop 2020-11-20T13:56:30.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 shutdown_connections 2020-11-20T13:56:30.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3de4051320 0x7f3de40537e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e040b0080 0x7f3e0413fef0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3e0409bfe0 0x7f3e040b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 --2- 172.21.15.117:0/945324533 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3e0409c7a0 0x7f3e0413fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 >> 172.21.15.117:0/945324533 conn(0x7f3e0405b9f0 msgr2=0x7f3e0409d030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 shutdown_connections 2020-11-20T13:56:30.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.333+0000 7f3e0ad7b700 1 -- 172.21.15.117:0/945324533 wait complete. 2020-11-20T13:56:30.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T13:56:30.344 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:30.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.454+0000 7f400f8ec700 1 Processor -- start 2020-11-20T13:56:30.454 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.454+0000 7f400f8ec700 1 -- start start 2020-11-20T13:56:30.457 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400f8ec700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f4008098d60 0x7f400809b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400f8ec700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40080b5240 0x7f40080630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400f8ec700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080b4860 con 0x7f40080635b0 2020-11-20T13:56:30.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400f8ec700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40081439f0 con 0x7f4008098d60 2020-11-20T13:56:30.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400f8ec700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40080b4b00 con 0x7f40080b5240 2020-11-20T13:56:30.458 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400de89700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f40080635b0 0x7f400809c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60722/0 (socket says 172.21.15.117:60722) 2020-11-20T13:56:30.459 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400de89700 1 -- 172.21.15.117:0/1098080727 learned_addr learned my addr 172.21.15.117:0/1098080727 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:30.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400d688700 1 --2- 172.21.15.117:0/1098080727 >> v2:172.21.15.85:3300/0 conn(0x7f4008098d60 0x7f400809b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400ce87700 1 --2- 172.21.15.117:0/1098080727 >> v2:172.21.15.117:3300/0 conn(0x7f40080b5240 0x7f40080630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f3ff7fff700 1 -- 172.21.15.117:0/1098080727 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332401030 0 0) 0x7f40080b4860 con 0x7f40080635b0 2020-11-20T13:56:30.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f3ff7fff700 1 -- 172.21.15.117:0/1098080727 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ff0004340 con 0x7f40080635b0 2020-11-20T13:56:30.460 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f3ff7fff700 1 -- 172.21.15.117:0/1098080727 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 504895182 0 0) 0x7f3ff0004340 con 0x7f40080635b0 2020-11-20T13:56:30.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400ce87700 1 -- 172.21.15.117:0/1098080727 >> v2:172.21.15.85:3300/0 conn(0x7f4008098d60 msgr2=0x7f400809b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400ce87700 1 --2- 172.21.15.117:0/1098080727 >> v2:172.21.15.85:3300/0 conn(0x7f4008098d60 0x7f400809b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400ce87700 1 -- 172.21.15.117:0/1098080727 >> v1:172.21.15.117:6789/0 conn(0x7f40080635b0 legacy=0x7f400809c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.461 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.455+0000 7f400ce87700 1 -- 172.21.15.117:0/1098080727 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40080b4da0 con 0x7f40080b5240 2020-11-20T13:56:30.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400ce87700 1 --2- 172.21.15.117:0/1098080727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080b5240 0x7f40080630c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f4000001bf0 tx=0x7f4004006dc0).ready entity=mon.1 client_cookie=8912f7b75440980b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f3ff7fff700 1 -- 172.21.15.117:0/1098080727 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4000001a50 con 0x7f40080b5240 2020-11-20T13:56:30.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f3ff7fff700 1 -- 172.21.15.117:0/1098080727 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400400d5c0 con 0x7f40080b5240 2020-11-20T13:56:30.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f3ff7fff700 1 -- 172.21.15.117:0/1098080727 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f400400d730 con 0x7f40080b5240 2020-11-20T13:56:30.462 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 -- 172.21.15.117:0/1098080727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080b5240 msgr2=0x7f40080630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1098080727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080b5240 0x7f40080630c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f4000001bf0 tx=0x7f4004006dc0).stop 2020-11-20T13:56:30.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 -- 172.21.15.117:0/1098080727 shutdown_connections 2020-11-20T13:56:30.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1098080727 >> v2:172.21.15.85:3300/0 conn(0x7f4008098d60 0x7f400809b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1098080727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080b5240 0x7f40080630c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.463 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 -- 172.21.15.117:0/1098080727 >> 172.21.15.117:0/1098080727 conn(0x7f40080b9770 msgr2=0x7f40080b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 -- 172.21.15.117:0/1098080727 shutdown_connections 2020-11-20T13:56:30.464 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.456+0000 7f400f8ec700 1 -- 172.21.15.117:0/1098080727 wait complete. 2020-11-20T13:56:30.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.457+0000 7f400f8ec700 1 Processor -- start 2020-11-20T13:56:30.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.457+0000 7f400f8ec700 1 -- start start 2020-11-20T13:56:30.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400f8ec700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080635b0 0x7f4008062910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400f8ec700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4008098d60 0x7f40081413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.468 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400f8ec700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40080b5240 0x7f40081418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400f8ec700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4008141d90 con 0x7f4008098d60 2020-11-20T13:56:30.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400f8ec700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081546f0 con 0x7f40080635b0 2020-11-20T13:56:30.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400f8ec700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4008154990 con 0x7f40080b5240 2020-11-20T13:56:30.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080635b0 0x7f4008062910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080635b0 0x7f4008062910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41568/0 (socket says 172.21.15.117:41568) 2020-11-20T13:56:30.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 -- 172.21.15.117:0/1677259183 learned_addr learned my addr 172.21.15.117:0/1677259183 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:30.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400ce87700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4008098d60 0x7f40081413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400de89700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40080b5240 0x7f40081418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 -- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40080b5240 msgr2=0x7f40081418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40080b5240 0x7f40081418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 -- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4008098d60 msgr2=0x7f40081413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4008098d60 0x7f40081413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 -- 172.21.15.117:0/1677259183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4008154c30 con 0x7f40080635b0 2020-11-20T13:56:30.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400ce87700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4008098d60 0x7f40081413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:30.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400d688700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080635b0 0x7f4008062910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f3ff800c000 tx=0x7f3ff8006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f3ff67fc700 1 -- 172.21.15.117:0/1677259183 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff800ca80 con 0x7f40080635b0 2020-11-20T13:56:30.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f3ff67fc700 1 -- 172.21.15.117:0/1677259183 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff800cbe0 con 0x7f40080635b0 2020-11-20T13:56:30.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f3ff67fc700 1 -- 172.21.15.117:0/1677259183 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ff80123f0 con 0x7f40080635b0 2020-11-20T13:56:30.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.458+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4008062e60 con 0x7f40080635b0 2020-11-20T13:56:30.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.459+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40081550d0 con 0x7f40080635b0 2020-11-20T13:56:30.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.459+0000 7f3ff67fc700 1 -- 172.21.15.117:0/1677259183 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3ff8012550 con 0x7f40080635b0 2020-11-20T13:56:30.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.460+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008057050 con 0x7f40080635b0 2020-11-20T13:56:30.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.460+0000 7f3ff67fc700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3fe80513c0 0x7f3fe8053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.460+0000 7f3ff67fc700 1 -- 172.21.15.117:0/1677259183 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f3ff8069340 con 0x7f40080635b0 2020-11-20T13:56:30.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.460+0000 7f400ce87700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3fe80513c0 0x7f3fe8053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.460+0000 7f400ce87700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3fe80513c0 0x7f3fe8053880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f4004018000 tx=0x7f4004015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.463+0000 7f3ff67fc700 1 -- 172.21.15.117:0/1677259183 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3ff8041730 con 0x7f40080635b0 2020-11-20T13:56:30.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.609+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4008154dc0 con 0x7f40080635b0 2020-11-20T13:56:30.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.609+0000 7f3ff67fc700 1 -- 172.21.15.117:0/1677259183 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v31) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3ff8044d50 con 0x7f40080635b0 2020-11-20T13:56:30.610 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:30.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3fe80513c0 msgr2=0x7f3fe8053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3fe80513c0 0x7f3fe8053880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f4004018000 tx=0x7f4004015000).stop 2020-11-20T13:56:30.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080635b0 msgr2=0x7f4008062910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080635b0 0x7f4008062910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f3ff800c000 tx=0x7f3ff8006dc0).stop 2020-11-20T13:56:30.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 shutdown_connections 2020-11-20T13:56:30.612 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3fe80513c0 0x7f3fe8053880 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f40080b5240 0x7f40081418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40080635b0 0x7f4008062910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 --2- 172.21.15.117:0/1677259183 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4008098d60 0x7f40081413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 >> 172.21.15.117:0/1677259183 conn(0x7f40080b9770 msgr2=0x7f40080b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.613 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 shutdown_connections 2020-11-20T13:56:30.614 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.611+0000 7f400f8ec700 1 -- 172.21.15.117:0/1677259183 wait complete. 2020-11-20T13:56:30.621 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.731+0000 7fd3c63cd700 1 Processor -- start 2020-11-20T13:56:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.731+0000 7fd3c63cd700 1 -- start start 2020-11-20T13:56:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c63cd700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd3c009cb40 0x7fd3c01437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c63cd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c0143c90 0x7fd3c00b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c63cd700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c00999d0 con 0x7fd3c009bd80 2020-11-20T13:56:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c63cd700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c00997f0 con 0x7fd3c009cb40 2020-11-20T13:56:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c63cd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0099c70 con 0x7fd3c0143c90 2020-11-20T13:56:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c0143c90 0x7fd3c00b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3bf7fe700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd3c009cb40 0x7fd3c01437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c0143c90 0x7fd3c00b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.117:41642/0 (socket says 172.21.15.117:41642) 2020-11-20T13:56:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 -- 172.21.15.117:0/3327562402 learned_addr learned my addr 172.21.15.117:0/3327562402 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 -- 172.21.15.117:0/3327562402 >> v2:172.21.15.85:3300/0 conn(0x7fd3c009cb40 msgr2=0x7fd3c01437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 --2- 172.21.15.117:0/3327562402 >> v2:172.21.15.85:3300/0 conn(0x7fd3c009cb40 0x7fd3c01437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 -- 172.21.15.117:0/3327562402 >> v1:172.21.15.85:6789/0 conn(0x7fd3c009bd80 legacy=0x7fd3c009a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 -- 172.21.15.117:0/3327562402 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c0144120 con 0x7fd3c0143c90 2020-11-20T13:56:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3bf7fe700 1 --2- 172.21.15.117:0/3327562402 >> v2:172.21.15.85:3300/0 conn(0x7fd3c009cb40 0x7fd3c01437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.732+0000 7fd3c496a700 1 --2- 172.21.15.117:0/3327562402 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c0143c90 0x7fd3c00b4110 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd3b80032b0 tx=0x7fd3b8009270).ready entity=mon.2 client_cookie=bf86ed66e87f266c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3beffd700 1 -- 172.21.15.117:0/3327562402 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b8009e60 con 0x7fd3c0143c90 2020-11-20T13:56:30.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3beffd700 1 -- 172.21.15.117:0/3327562402 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b800bc00 con 0x7fd3c0143c90 2020-11-20T13:56:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3beffd700 1 -- 172.21.15.117:0/3327562402 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b800bd70 con 0x7fd3c0143c90 2020-11-20T13:56:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 -- 172.21.15.117:0/3327562402 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c0143c90 msgr2=0x7fd3c00b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/3327562402 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c0143c90 0x7fd3c00b4110 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd3b80032b0 tx=0x7fd3b8009270).stop 2020-11-20T13:56:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 -- 172.21.15.117:0/3327562402 shutdown_connections 2020-11-20T13:56:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/3327562402 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c0143c90 0x7fd3c00b4110 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.742 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/3327562402 >> v2:172.21.15.85:3300/0 conn(0x7fd3c009cb40 0x7fd3c01437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.742 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 -- 172.21.15.117:0/3327562402 >> 172.21.15.117:0/3327562402 conn(0x7fd3c00b9770 msgr2=0x7fd3c00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 -- 172.21.15.117:0/3327562402 shutdown_connections 2020-11-20T13:56:30.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.733+0000 7fd3c63cd700 1 -- 172.21.15.117:0/3327562402 wait complete. 2020-11-20T13:56:30.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.734+0000 7fd3c63cd700 1 Processor -- start 2020-11-20T13:56:30.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.734+0000 7fd3c63cd700 1 -- start start 2020-11-20T13:56:30.743 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3c63cd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c009bd80 0x7fd3c0067700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3c63cd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3c009cb40 0x7fd3c0144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3c63cd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3c0143c90 0x7fd3c0154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3c63cd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0154140 con 0x7fd3c0143c90 2020-11-20T13:56:30.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3c63cd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0142580 con 0x7fd3c009cb40 2020-11-20T13:56:30.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3c63cd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0154330 con 0x7fd3c009bd80 2020-11-20T13:56:30.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3c009cb40 0x7fd3c0144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3c009cb40 0x7fd3c0144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41584/0 (socket says 172.21.15.117:41584) 2020-11-20T13:56:30.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 -- 172.21.15.117:0/807621440 learned_addr learned my addr 172.21.15.117:0/807621440 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:30.745 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bffff700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c009bd80 0x7fd3c0067700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3c496a700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3c0143c90 0x7fd3c0154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 -- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c009bd80 msgr2=0x7fd3c0067700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c009bd80 0x7fd3c0067700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 -- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3c0143c90 msgr2=0x7fd3c0154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3c0143c90 0x7fd3c0154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 -- 172.21.15.117:0/807621440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c01545d0 con 0x7fd3c009cb40 2020-11-20T13:56:30.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bf7fe700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3c009cb40 0x7fd3c0144a90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fd3c00b26e0 tx=0x7fd3b4006dc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.735+0000 7fd3bd7fa700 1 -- 172.21.15.117:0/807621440 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b400cda0 con 0x7fd3c009cb40 2020-11-20T13:56:30.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.736+0000 7fd3bd7fa700 1 -- 172.21.15.117:0/807621440 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b400eed0 con 0x7fd3c009cb40 2020-11-20T13:56:30.747 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.736+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c005fce0 con 0x7fd3c009cb40 2020-11-20T13:56:30.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.736+0000 7fd3bd7fa700 1 -- 172.21.15.117:0/807621440 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b40125e0 con 0x7fd3c009cb40 2020-11-20T13:56:30.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.736+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c00b3090 con 0x7fd3c009cb40 2020-11-20T13:56:30.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.737+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c0154760 con 0x7fd3c009cb40 2020-11-20T13:56:30.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.737+0000 7fd3bd7fa700 1 -- 172.21.15.117:0/807621440 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd3b4012740 con 0x7fd3c009cb40 2020-11-20T13:56:30.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.737+0000 7fd3bd7fa700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3a0051290 0x7fd3a0053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:30.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.737+0000 7fd3bd7fa700 1 -- 172.21.15.117:0/807621440 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7fd3b40695a0 con 0x7fd3c009cb40 2020-11-20T13:56:30.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.737+0000 7fd3bffff700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3a0051290 0x7fd3a0053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:30.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.737+0000 7fd3bffff700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3a0051290 0x7fd3a0053750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd3b000b000 tx=0x7fd3b0009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:30.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.740+0000 7fd3bd7fa700 1 -- 172.21.15.117:0/807621440 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd3b4041b00 con 0x7fd3c009cb40 2020-11-20T13:56:30.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.892+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3c0057050 con 0x7fd3c009cb40 2020-11-20T13:56:30.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.892+0000 7fd3bd7fa700 1 -- 172.21.15.117:0/807621440 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd3b4045120 con 0x7fd3c009cb40 2020-11-20T13:56:30.893 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:30.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.893+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3a0051290 msgr2=0x7fd3a0053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.893+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3a0051290 0x7fd3a0053750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd3b000b000 tx=0x7fd3b0009040).stop 2020-11-20T13:56:30.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.893+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3c009cb40 msgr2=0x7fd3c0144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:30.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.893+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3c009cb40 0x7fd3c0144a90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fd3c00b26e0 tx=0x7fd3b4006dc0).stop 2020-11-20T13:56:30.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 shutdown_connections 2020-11-20T13:56:30.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd3a0051290 0x7fd3a0053750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd3c009bd80 0x7fd3c0067700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3c0143c90 0x7fd3c0154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 --2- 172.21.15.117:0/807621440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3c009cb40 0x7fd3c0144a90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:30.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 >> 172.21.15.117:0/807621440 conn(0x7fd3c00b9770 msgr2=0x7fd3c0062cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:30.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 shutdown_connections 2020-11-20T13:56:30.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:30.894+0000 7fd3c63cd700 1 -- 172.21.15.117:0/807621440 wait complete. 2020-11-20T13:56:30.904 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-11-20T13:56:31.012 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f358e817700 1 Processor -- start 2020-11-20T13:56:31.013 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f358e817700 1 -- start start 2020-11-20T13:56:31.013 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f358e817700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3588147190 0x7f3588148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.013 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f358e817700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f3588061f40 0x7f358814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.014 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f358e817700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3588147880 con 0x7f358809aea0 2020-11-20T13:56:31.014 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f358e817700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3588147b20 con 0x7f3588061f40 2020-11-20T13:56:31.014 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f358e817700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3588147dc0 con 0x7f3588147190 2020-11-20T13:56:31.014 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f3587fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3588147190 0x7f3588148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f3587fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3588147190 0x7f3588148260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41590/0 (socket says 172.21.15.117:41590) 2020-11-20T13:56:31.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.010+0000 7f35877fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f358809aea0 0x7f35880b8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60752/0 (socket says 172.21.15.117:60752) 2020-11-20T13:56:31.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3587fff700 1 -- 172.21.15.117:0/4181216959 learned_addr learned my addr 172.21.15.117:0/4181216959 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:31.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f358cdb4700 1 --2- 172.21.15.117:0/4181216959 >> v2:172.21.15.85:3300/0 conn(0x7f3588061f40 0x7f358814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.015 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3587fff700 1 -- 172.21.15.117:0/4181216959 >> v2:172.21.15.85:3300/0 conn(0x7f3588061f40 msgr2=0x7f358814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3587fff700 1 --2- 172.21.15.117:0/4181216959 >> v2:172.21.15.85:3300/0 conn(0x7f3588061f40 0x7f358814acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3587fff700 1 -- 172.21.15.117:0/4181216959 >> v1:172.21.15.117:6789/0 conn(0x7f358809aea0 legacy=0x7f35880b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3586ffd700 1 -- 172.21.15.117:0/4181216959 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895382675 0 0) 0x7f3588147880 con 0x7f358809aea0 2020-11-20T13:56:31.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3587fff700 1 -- 172.21.15.117:0/4181216959 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588147570 con 0x7f3588147190 2020-11-20T13:56:31.016 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f358cdb4700 1 --2- 172.21.15.117:0/4181216959 >> v2:172.21.15.85:3300/0 conn(0x7f3588061f40 0x7f358814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:31.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3587fff700 1 --2- 172.21.15.117:0/4181216959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588148260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3578002f30 tx=0x7f3578006e80).ready entity=mon.1 client_cookie=a790a67f7506186d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3586ffd700 1 -- 172.21.15.117:0/4181216959 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35780097f0 con 0x7f3588147190 2020-11-20T13:56:31.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3586ffd700 1 -- 172.21.15.117:0/4181216959 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3578009950 con 0x7f3588147190 2020-11-20T13:56:31.017 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f3586ffd700 1 -- 172.21.15.117:0/4181216959 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3578009cb0 con 0x7f3588147190 2020-11-20T13:56:31.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f358e817700 1 -- 172.21.15.117:0/4181216959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 msgr2=0x7f3588148260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.011+0000 7f358e817700 1 --2- 172.21.15.117:0/4181216959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588148260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f3578002f30 tx=0x7f3578006e80).stop 2020-11-20T13:56:31.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.012+0000 7f358e817700 1 -- 172.21.15.117:0/4181216959 shutdown_connections 2020-11-20T13:56:31.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.012+0000 7f358e817700 1 --2- 172.21.15.117:0/4181216959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588148260 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.012+0000 7f358e817700 1 --2- 172.21.15.117:0/4181216959 >> v2:172.21.15.85:3300/0 conn(0x7f3588061f40 0x7f358814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.012+0000 7f358e817700 1 -- 172.21.15.117:0/4181216959 >> 172.21.15.117:0/4181216959 conn(0x7f358805c4c0 msgr2=0x7f35880b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.012+0000 7f358e817700 1 -- 172.21.15.117:0/4181216959 shutdown_connections 2020-11-20T13:56:31.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.012+0000 7f358e817700 1 -- 172.21.15.117:0/4181216959 wait complete. 2020-11-20T13:56:31.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.012+0000 7f358e817700 1 Processor -- start 2020-11-20T13:56:31.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358e817700 1 -- start start 2020-11-20T13:56:31.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358e817700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3588061f40 0x7f358805fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358e817700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f358809aea0 0x7f35880b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358e817700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358e817700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b46e0 con 0x7f3588061f40 2020-11-20T13:56:31.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358e817700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b3ab0 con 0x7f3588147190 2020-11-20T13:56:31.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358e817700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b3d50 con 0x7f358809aea0 2020-11-20T13:56:31.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358cdb4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358cdb4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588142b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41600/0 (socket says 172.21.15.117:41600) 2020-11-20T13:56:31.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.013+0000 7f358cdb4700 1 -- 172.21.15.117:0/3114089124 learned_addr learned my addr 172.21.15.117:0/3114089124 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:31.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f3587fff700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3588061f40 0x7f358805fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f35877fe700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f358809aea0 0x7f35880b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358cdb4700 1 -- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f358809aea0 msgr2=0x7f35880b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358cdb4700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f358809aea0 0x7f35880b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358cdb4700 1 -- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3588061f40 msgr2=0x7f358805fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358cdb4700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3588061f40 0x7f358805fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358cdb4700 1 -- 172.21.15.117:0/3114089124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880b3790 con 0x7f3588147190 2020-11-20T13:56:31.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358cdb4700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588142b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f3580006dd0 tx=0x7f3580008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f35857fa700 1 -- 172.21.15.117:0/3114089124 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f358000c810 con 0x7f3588147190 2020-11-20T13:56:31.025 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f35857fa700 1 -- 172.21.15.117:0/3114089124 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000c970 con 0x7f3588147190 2020-11-20T13:56:31.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35880b57c0 con 0x7f3588147190 2020-11-20T13:56:31.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f35857fa700 1 -- 172.21.15.117:0/3114089124 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f358000ccd0 con 0x7f3588147190 2020-11-20T13:56:31.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.014+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35880b3170 con 0x7f3588147190 2020-11-20T13:56:31.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.015+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588057050 con 0x7f3588147190 2020-11-20T13:56:31.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.015+0000 7f35857fa700 1 -- 172.21.15.117:0/3114089124 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f358001d460 con 0x7f3588147190 2020-11-20T13:56:31.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.016+0000 7f35857fa700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3568051210 0x7f35680536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.016+0000 7f35857fa700 1 -- 172.21.15.117:0/3114089124 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f35800691d0 con 0x7f3588147190 2020-11-20T13:56:31.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.016+0000 7f3587fff700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3568051210 0x7f35680536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.016+0000 7f3587fff700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3568051210 0x7f35680536d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f357801c000 tx=0x7f3578015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.019+0000 7f35857fa700 1 -- 172.21.15.117:0/3114089124 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f35800416d0 con 0x7f3588147190 2020-11-20T13:56:31.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.172+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f35880b68a0 con 0x7f3588147190 2020-11-20T13:56:31.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.172+0000 7f35857fa700 1 -- 172.21.15.117:0/3114089124 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3580044cf0 con 0x7f3588147190 2020-11-20T13:56:31.173 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:31.173 INFO:teuthology.orchestra.run.smithi117.stdout:[ 2020-11-20T13:56:31.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:31.174 INFO:teuthology.orchestra.run.smithi117.stdout: "poolnum": 1, 2020-11-20T13:56:31.174 INFO:teuthology.orchestra.run.smithi117.stdout: "poolname": "device_health_metrics" 2020-11-20T13:56:31.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:31.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2020-11-20T13:56:31.175 INFO:teuthology.orchestra.run.smithi117.stdout: "poolnum": 2, 2020-11-20T13:56:31.175 INFO:teuthology.orchestra.run.smithi117.stdout: "poolname": "rbd" 2020-11-20T13:56:31.175 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:31.175 INFO:teuthology.orchestra.run.smithi117.stdout:] 2020-11-20T13:56:31.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3568051210 msgr2=0x7f35680536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.176 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3568051210 0x7f35680536d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f357801c000 tx=0x7f3578015000).stop 2020-11-20T13:56:31.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 msgr2=0x7f3588142b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588142b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f3580006dd0 tx=0x7f3580008040).stop 2020-11-20T13:56:31.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 shutdown_connections 2020-11-20T13:56:31.177 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3568051210 0x7f35680536d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3588147190 0x7f3588142b60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3588061f40 0x7f358805fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 --2- 172.21.15.117:0/3114089124 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f358809aea0 0x7f35880b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 >> 172.21.15.117:0/3114089124 conn(0x7f358805c4c0 msgr2=0x7f3588144310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 shutdown_connections 2020-11-20T13:56:31.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.174+0000 7f358e817700 1 -- 172.21.15.117:0/3114089124 wait complete. 2020-11-20T13:56:31.184 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T13:56:31.185 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-11-20T13:56:31.293 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.293+0000 7f25ff6ef700 1 Processor -- start 2020-11-20T13:56:31.294 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.293+0000 7f25ff6ef700 1 -- start start 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25ff6ef700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809bd80 0x7f25f809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25ff6ef700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25f809a110 0x7f25f81437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25ff6ef700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f80999d0 con 0x7f25f8143c90 2020-11-20T13:56:31.296 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25ff6ef700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f8099c70 con 0x7f25f809a110 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25ff6ef700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f8144120 con 0x7f25f809bd80 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25f809a110 0x7f25f81437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25f809a110 0x7f25f81437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41604/0 (socket says 172.21.15.117:41604) 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 -- 172.21.15.117:0/210186701 learned_addr learned my addr 172.21.15.117:0/210186701 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:31.297 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fd48b700 1 --2- 172.21.15.117:0/210186701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809bd80 0x7f25f809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 -- 172.21.15.117:0/210186701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809bd80 msgr2=0x7f25f809cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 --2- 172.21.15.117:0/210186701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809bd80 0x7f25f809cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 -- 172.21.15.117:0/210186701 >> v1:172.21.15.85:6789/0 conn(0x7f25f8143c90 legacy=0x7f25f80b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 -- 172.21.15.117:0/210186701 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f809c160 con 0x7f25f809a110 2020-11-20T13:56:31.298 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.294+0000 7f25fcc8a700 1 --2- 172.21.15.117:0/210186701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f809a110 0x7f25f81437a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f25f809b9c0 tx=0x7f25f4006e80).ready entity=mon.1 client_cookie=70845f5ec312ea1f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25e7fff700 1 -- 172.21.15.117:0/210186701 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f4009770 con 0x7f25f809a110 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25e7fff700 1 -- 172.21.15.117:0/210186701 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f40098d0 con 0x7f25f809a110 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25e7fff700 1 -- 172.21.15.117:0/210186701 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f4009a40 con 0x7f25f809a110 2020-11-20T13:56:31.299 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 -- 172.21.15.117:0/210186701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f809a110 msgr2=0x7f25f81437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/210186701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f809a110 0x7f25f81437a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f25f809b9c0 tx=0x7f25f4006e80).stop 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 -- 172.21.15.117:0/210186701 shutdown_connections 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/210186701 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809bd80 0x7f25f809cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/210186701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f809a110 0x7f25f81437a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.300 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 -- 172.21.15.117:0/210186701 >> 172.21.15.117:0/210186701 conn(0x7f25f80b9770 msgr2=0x7f25f80b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:31.301 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 -- 172.21.15.117:0/210186701 shutdown_connections 2020-11-20T13:56:31.301 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.295+0000 7f25ff6ef700 1 -- 172.21.15.117:0/210186701 wait complete. 2020-11-20T13:56:31.303 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.296+0000 7f25ff6ef700 1 Processor -- start 2020-11-20T13:56:31.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.296+0000 7f25ff6ef700 1 -- start start 2020-11-20T13:56:31.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25ff6ef700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809a110 0x7f25f8067070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25ff6ef700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f25f809bd80 0x7f25f8157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25ff6ef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8143c90 0x7f25f80aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.304 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25ff6ef700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f80b13e0 con 0x7f25f809bd80 2020-11-20T13:56:31.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25ff6ef700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f8157670 con 0x7f25f8143c90 2020-11-20T13:56:31.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25ff6ef700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f8157910 con 0x7f25f809a110 2020-11-20T13:56:31.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8143c90 0x7f25f80aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.305 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8143c90 0x7f25f80aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41612/0 (socket says 172.21.15.117:41612) 2020-11-20T13:56:31.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 -- 172.21.15.117:0/2601650214 learned_addr learned my addr 172.21.15.117:0/2601650214 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:31.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fd48b700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809a110 0x7f25f8067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fcc8a700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f25f809bd80 0x7f25f8157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 -- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809a110 msgr2=0x7f25f8067070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.306 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809a110 0x7f25f8067070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 -- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f25f809bd80 msgr2=0x7f25f8157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:31.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f25f809bd80 0x7f25f8157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:31.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.297+0000 7f25fdc8c700 1 -- 172.21.15.117:0/2601650214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80b33a0 con 0x7f25f8143c90 2020-11-20T13:56:31.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.298+0000 7f25fdc8c700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8143c90 0x7f25f80aec60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f25f00022d0 tx=0x7f25f0007f10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.307 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.298+0000 7f25e67fc700 1 -- 172.21.15.117:0/2601650214 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f000a5a0 con 0x7f25f8143c90 2020-11-20T13:56:31.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.298+0000 7f25e67fc700 1 -- 172.21.15.117:0/2601650214 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f000a700 con 0x7f25f8143c90 2020-11-20T13:56:31.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.298+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f81582a0 con 0x7f25f8143c90 2020-11-20T13:56:31.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.298+0000 7f25e67fc700 1 -- 172.21.15.117:0/2601650214 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25f000aa60 con 0x7f25f8143c90 2020-11-20T13:56:31.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.298+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f80b1110 con 0x7f25f8143c90 2020-11-20T13:56:31.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.299+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f80b3590 con 0x7f25f8143c90 2020-11-20T13:56:31.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.299+0000 7f25e67fc700 1 -- 172.21.15.117:0/2601650214 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f25f0019030 con 0x7f25f8143c90 2020-11-20T13:56:31.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.299+0000 7f25e67fc700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f25d8051320 0x7f25d80537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:31.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.299+0000 7f25e67fc700 1 -- 172.21.15.117:0/2601650214 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5600+0+0 (secure 0 0 0) 0x7f25f0068810 con 0x7f25f8143c90 2020-11-20T13:56:31.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.299+0000 7f25fd48b700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f25d8051320 0x7f25d80537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:31.309 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.299+0000 7f25fd48b700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f25d8051320 0x7f25d80537e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f25e800c000 tx=0x7f25e8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:31.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.302+0000 7f25e67fc700 1 -- 172.21.15.117:0/2601650214 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f25f0040be0 con 0x7f25f8143c90 2020-11-20T13:56:31.452 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:31.451+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f25f8157ad0 con 0x7f25f8143c90 2020-11-20T13:56:32.057 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.057+0000 7f25e67fc700 1 -- 172.21.15.117:0/2601650214 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f25f0044200 con 0x7f25f8143c90 2020-11-20T13:56:32.059 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f25d8051320 msgr2=0x7f25d80537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:32.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f25d8051320 0x7f25d80537e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f25e800c000 tx=0x7f25e8008040).stop 2020-11-20T13:56:32.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8143c90 msgr2=0x7f25f80aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:32.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8143c90 0x7f25f80aec60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f25f00022d0 tx=0x7f25f0007f10).stop 2020-11-20T13:56:32.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 shutdown_connections 2020-11-20T13:56:32.060 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f25d8051320 0x7f25d80537e0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0x7f25e800c000 tx=0x7f25e8008040).stop 2020-11-20T13:56:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f25f809a110 0x7f25f8067070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f8143c90 0x7f25f80aec60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 --2- 172.21.15.117:0/2601650214 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f25f809bd80 0x7f25f8157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.061 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 >> 172.21.15.117:0/2601650214 conn(0x7f25f80b9770 msgr2=0x7f25f80b5490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 shutdown_connections 2020-11-20T13:56:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.059+0000 7f25ff6ef700 1 -- 172.21.15.117:0/2601650214 wait complete. 2020-11-20T13:56:32.062 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' created 2020-11-20T13:56:32.070 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-11-20T13:56:32.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.180+0000 7f49cd021700 1 Processor -- start 2020-11-20T13:56:32.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.180+0000 7f49cd021700 1 -- start start 2020-11-20T13:56:32.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49cd021700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f49c809bd80 0x7f49c809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:32.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49cd021700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c81437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:32.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49cd021700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49c8143c90 0x7f49c80b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:32.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49cd021700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c80999d0 con 0x7f49c809bd80 2020-11-20T13:56:32.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49cd021700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c8099c70 con 0x7f49c8143c90 2020-11-20T13:56:32.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49cd021700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c8144120 con 0x7f49c809a110 2020-11-20T13:56:32.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49c8143c90 0x7f49c80b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:32.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49c8143c90 0x7f49c80b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41622/0 (socket says 172.21.15.117:41622) 2020-11-20T13:56:32.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 -- 172.21.15.117:0/2538749978 learned_addr learned my addr 172.21.15.117:0/2538749978 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:32.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49bffff700 1 --2- 172.21.15.117:0/2538749978 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c81437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:32.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c6d9d700 1 --2- 172.21.15.117:0/2538749978 >> v2:172.21.15.85:3300/0 conn(0x7f49c809bd80 0x7f49c809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:32.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 -- 172.21.15.117:0/2538749978 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 msgr2=0x7f49c81437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:32.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 --2- 172.21.15.117:0/2538749978 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c81437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 -- 172.21.15.117:0/2538749978 >> v2:172.21.15.85:3300/0 conn(0x7f49c809bd80 msgr2=0x7f49c809cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:32.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 --2- 172.21.15.117:0/2538749978 >> v2:172.21.15.85:3300/0 conn(0x7f49c809bd80 0x7f49c809cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 -- 172.21.15.117:0/2538749978 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c809c160 con 0x7f49c8143c90 2020-11-20T13:56:32.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.181+0000 7f49c759e700 1 --2- 172.21.15.117:0/2538749978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c8143c90 0x7f49c80b4110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f49c00032b0 tx=0x7f49c0009270).ready entity=mon.1 client_cookie=d2a15ce77532b9f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:32.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49c659c700 1 -- 172.21.15.117:0/2538749978 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c000bc00 con 0x7f49c8143c90 2020-11-20T13:56:32.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49c659c700 1 -- 172.21.15.117:0/2538749978 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000bd60 con 0x7f49c8143c90 2020-11-20T13:56:32.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49c659c700 1 -- 172.21.15.117:0/2538749978 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49c0011400 con 0x7f49c8143c90 2020-11-20T13:56:32.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 -- 172.21.15.117:0/2538749978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c8143c90 msgr2=0x7f49c80b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:32.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 --2- 172.21.15.117:0/2538749978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c8143c90 0x7f49c80b4110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f49c00032b0 tx=0x7f49c0009270).stop 2020-11-20T13:56:32.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 -- 172.21.15.117:0/2538749978 shutdown_connections 2020-11-20T13:56:32.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 --2- 172.21.15.117:0/2538749978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c8143c90 0x7f49c80b4110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 --2- 172.21.15.117:0/2538749978 >> v2:172.21.15.85:3300/0 conn(0x7f49c809bd80 0x7f49c809cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 --2- 172.21.15.117:0/2538749978 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c81437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 -- 172.21.15.117:0/2538749978 >> 172.21.15.117:0/2538749978 conn(0x7f49c80b9770 msgr2=0x7f49c80b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:32.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 -- 172.21.15.117:0/2538749978 shutdown_connections 2020-11-20T13:56:32.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.182+0000 7f49cd021700 1 -- 172.21.15.117:0/2538749978 wait complete. 2020-11-20T13:56:32.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.183+0000 7f49cd021700 1 Processor -- start 2020-11-20T13:56:32.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.183+0000 7f49cd021700 1 -- start start 2020-11-20T13:56:32.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c813bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:32.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c809bd80 0x7f49c809ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:32.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49c8143c90 0x7f49c8139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:32.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813c1c0 con 0x7f49c8143c90 2020-11-20T13:56:32.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813b590 con 0x7f49c809bd80 2020-11-20T13:56:32.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c813b830 con 0x7f49c809a110 2020-11-20T13:56:32.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c809bd80 0x7f49c809ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:32.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c809bd80 0x7f49c809ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41626/0 (socket says 172.21.15.117:41626) 2020-11-20T13:56:32.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 -- 172.21.15.117:0/3314147738 learned_addr learned my addr 172.21.15.117:0/3314147738 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:32.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49c6d9d700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c813bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:32.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49c759e700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49c8143c90 0x7f49c8139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:32.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 -- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 msgr2=0x7f49c813bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:32.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c813bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 -- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49c8143c90 msgr2=0x7f49c8139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:32.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49c8143c90 0x7f49c8139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:32.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 -- 172.21.15.117:0/3314147738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c809f350 con 0x7f49c809bd80 2020-11-20T13:56:32.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49bffff700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c809bd80 0x7f49c809ea40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f49b0003070 tx=0x7f49b0008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:32.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49c4d99700 1 -- 172.21.15.117:0/3314147738 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b0009790 con 0x7f49c809bd80 2020-11-20T13:56:32.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49c4d99700 1 -- 172.21.15.117:0/3314147738 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b00098f0 con 0x7f49c809bd80 2020-11-20T13:56:32.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49c4d99700 1 -- 172.21.15.117:0/3314147738 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49b0009c50 con 0x7f49c809bd80 2020-11-20T13:56:32.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c813d7d0 con 0x7f49c809bd80 2020-11-20T13:56:32.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.184+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c809f0f0 con 0x7f49c809bd80 2020-11-20T13:56:32.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.185+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c8057050 con 0x7f49c809bd80 2020-11-20T13:56:32.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.186+0000 7f49c4d99700 1 -- 172.21.15.117:0/3314147738 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f49b0013480 con 0x7f49c809bd80 2020-11-20T13:56:32.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.186+0000 7f49c4d99700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f49ac051210 0x7f49ac0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:32.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.186+0000 7f49c4d99700 1 -- 172.21.15.117:0/3314147738 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5954+0+0 (secure 0 0 0) 0x7f49b0069510 con 0x7f49c809bd80 2020-11-20T13:56:32.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.186+0000 7f49c6d9d700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f49ac051210 0x7f49ac0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:32.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.186+0000 7f49c6d9d700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f49ac051210 0x7f49ac0536d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f49b800c000 tx=0x7f49b8008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:32.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.189+0000 7f49c4d99700 1 -- 172.21.15.117:0/3314147738 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f49b00418a0 con 0x7f49c809bd80 2020-11-20T13:56:32.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:32.341+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f49c813b9f0 con 0x7f49c809bd80 2020-11-20T13:56:33.065 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.064+0000 7f49c4d99700 1 -- 172.21.15.117:0/3314147738 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v33) v1 ==== 125+0+0 (secure 0 0 0) 0x7f49b0009db0 con 0x7f49c809bd80 2020-11-20T13:56:33.067 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f49ac051210 msgr2=0x7f49ac0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f49ac051210 0x7f49ac0536d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f49b800c000 tx=0x7f49b8008040).stop 2020-11-20T13:56:33.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c809bd80 msgr2=0x7f49c809ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c809bd80 0x7f49c809ea40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f49b0003070 tx=0x7f49b0008040).stop 2020-11-20T13:56:33.068 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 shutdown_connections 2020-11-20T13:56:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f49ac051210 0x7f49ac0536d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f49c809a110 0x7f49c813bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f49c8143c90 0x7f49c8139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 --2- 172.21.15.117:0/3314147738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c809bd80 0x7f49c809ea40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.069 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 >> 172.21.15.117:0/3314147738 conn(0x7f49c80b9770 msgr2=0x7f49c805f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 shutdown_connections 2020-11-20T13:56:33.070 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.067+0000 7f49cd021700 1 -- 172.21.15.117:0/3314147738 wait complete. 2020-11-20T13:56:33.070 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' created 2020-11-20T13:56:33.080 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T13:56:33.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f1751a6e700 1 Processor -- start 2020-11-20T13:56:33.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f1751a6e700 1 -- start start 2020-11-20T13:56:33.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f1751a6e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09bd80 0x7f174c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f1751a6e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f174c143eb0 0x7f174c0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f1751a6e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174c0999d0 con 0x7f174c09a110 2020-11-20T13:56:33.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f1751a6e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174c099c70 con 0x7f174c143eb0 2020-11-20T13:56:33.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f1751a6e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c09c160 con 0x7f174c09bd80 2020-11-20T13:56:33.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f174bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f174c143eb0 0x7f174c0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f174affd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f174c09a110 0x7f174c1437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60792/0 (socket says 172.21.15.117:60792) 2020-11-20T13:56:33.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.193+0000 7f174affd700 1 -- 172.21.15.117:0/3438390576 learned_addr learned my addr 172.21.15.117:0/3438390576 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:33.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174b7fe700 1 --2- 172.21.15.117:0/3438390576 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09bd80 0x7f174c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174bfff700 1 -- 172.21.15.117:0/3438390576 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09bd80 msgr2=0x7f174c09cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174bfff700 1 --2- 172.21.15.117:0/3438390576 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09bd80 0x7f174c09cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174bfff700 1 -- 172.21.15.117:0/3438390576 >> v1:172.21.15.117:6789/0 conn(0x7f174c09a110 legacy=0x7f174c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174a7fc700 1 -- 172.21.15.117:0/3438390576 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204582381 0 0) 0x7f174c0999d0 con 0x7f174c09a110 2020-11-20T13:56:33.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174bfff700 1 -- 172.21.15.117:0/3438390576 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c098cb0 con 0x7f174c143eb0 2020-11-20T13:56:33.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174bfff700 1 --2- 172.21.15.117:0/3438390576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c143eb0 0x7f174c0b4110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f17440032b0 tx=0x7f17440091a0).ready entity=mon.1 client_cookie=ba2f87e0a4483636 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174a7fc700 1 -- 172.21.15.117:0/3438390576 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f174400ba50 con 0x7f174c143eb0 2020-11-20T13:56:33.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174a7fc700 1 -- 172.21.15.117:0/3438390576 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174400bbb0 con 0x7f174c143eb0 2020-11-20T13:56:33.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f174a7fc700 1 -- 172.21.15.117:0/3438390576 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f174400bd20 con 0x7f174c143eb0 2020-11-20T13:56:33.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f1751a6e700 1 -- 172.21.15.117:0/3438390576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c143eb0 msgr2=0x7f174c0b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3438390576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c143eb0 0x7f174c0b4110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f17440032b0 tx=0x7f17440091a0).stop 2020-11-20T13:56:33.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f1751a6e700 1 -- 172.21.15.117:0/3438390576 shutdown_connections 2020-11-20T13:56:33.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.194+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3438390576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c143eb0 0x7f174c0b4110 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.195+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3438390576 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09bd80 0x7f174c09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.195+0000 7f1751a6e700 1 -- 172.21.15.117:0/3438390576 >> 172.21.15.117:0/3438390576 conn(0x7f174c0b9770 msgr2=0x7f174c0b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.195+0000 7f1751a6e700 1 -- 172.21.15.117:0/3438390576 shutdown_connections 2020-11-20T13:56:33.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.195+0000 7f1751a6e700 1 -- 172.21.15.117:0/3438390576 wait complete. 2020-11-20T13:56:33.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.195+0000 7f1751a6e700 1 Processor -- start 2020-11-20T13:56:33.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f1751a6e700 1 -- start start 2020-11-20T13:56:33.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f1751a6e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09a110 0x7f174c09f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f1751a6e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c09bd80 0x7f174c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f1751a6e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f174c143eb0 0x7f174c066b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f1751a6e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0674e0 con 0x7f174c143eb0 2020-11-20T13:56:33.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f1751a6e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0676f0 con 0x7f174c09bd80 2020-11-20T13:56:33.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f1751a6e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0aeeb0 con 0x7f174c09a110 2020-11-20T13:56:33.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f174affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c09bd80 0x7f174c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f174affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c09bd80 0x7f174c0b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41638/0 (socket says 172.21.15.117:41638) 2020-11-20T13:56:33.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.196+0000 7f174affd700 1 -- 172.21.15.117:0/3745275736 learned_addr learned my addr 172.21.15.117:0/3745275736 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:33.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174b7fe700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09a110 0x7f174c09f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174bfff700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f174c143eb0 0x7f174c066b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174affd700 1 -- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09a110 msgr2=0x7f174c09f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174affd700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09a110 0x7f174c09f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174affd700 1 -- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f174c143eb0 msgr2=0x7f174c066b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174affd700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f174c143eb0 0x7f174c066b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174affd700 1 -- 172.21.15.117:0/3745275736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c0b1790 con 0x7f174c09bd80 2020-11-20T13:56:33.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f174affd700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c09bd80 0x7f174c0b12a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1740000b80 tx=0x7f17400079b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f1748ff9700 1 -- 172.21.15.117:0/3745275736 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17400072e0 con 0x7f174c09bd80 2020-11-20T13:56:33.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f1748ff9700 1 -- 172.21.15.117:0/3745275736 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740007440 con 0x7f174c09bd80 2020-11-20T13:56:33.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f1748ff9700 1 -- 172.21.15.117:0/3745275736 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17400057d0 con 0x7f174c09bd80 2020-11-20T13:56:33.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c0b3390 con 0x7f174c09bd80 2020-11-20T13:56:33.210 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.197+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c0671f0 con 0x7f174c09bd80 2020-11-20T13:56:33.211 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.198+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174c0af070 con 0x7f174c09bd80 2020-11-20T13:56:33.211 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.198+0000 7f1748ff9700 1 -- 172.21.15.117:0/3745275736 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1740010070 con 0x7f174c09bd80 2020-11-20T13:56:33.211 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.198+0000 7f1748ff9700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f172c051320 0x7f172c0537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.211 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.198+0000 7f1748ff9700 1 -- 172.21.15.117:0/3745275736 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f17400685d0 con 0x7f174c09bd80 2020-11-20T13:56:33.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.198+0000 7f174b7fe700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f172c051320 0x7f172c0537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.199+0000 7f174b7fe700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f172c051320 0x7f172c0537e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f173c00d000 tx=0x7f173c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.202+0000 7f1748ff9700 1 -- 172.21.15.117:0/3745275736 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1740040870 con 0x7f174c09bd80 2020-11-20T13:56:33.416 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.415+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f174c057050 con 0x7f174c09bd80 2020-11-20T13:56:33.421 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.421+0000 7f1748ff9700 1 -- 172.21.15.117:0/3745275736 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v9) v1 ==== 187+0+0 (secure 0 0 0) 0x7f1740018070 con 0x7f174c09bd80 2020-11-20T13:56:33.427 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.427+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f172c051320 msgr2=0x7f172c0537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.427+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f172c051320 0x7f172c0537e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f173c00d000 tx=0x7f173c008040).stop 2020-11-20T13:56:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.427+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c09bd80 msgr2=0x7f174c0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.427+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c09bd80 0x7f174c0b12a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1740000b80 tx=0x7f17400079b0).stop 2020-11-20T13:56:33.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.432+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 shutdown_connections 2020-11-20T13:56:33.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.432+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f172c051320 0x7f172c0537e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.432+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f174c09a110 0x7f174c09f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.432+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f174c143eb0 0x7f174c066b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.432+0000 7f1751a6e700 1 --2- 172.21.15.117:0/3745275736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f174c09bd80 0x7f174c0b12a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.432+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 >> 172.21.15.117:0/3745275736 conn(0x7f174c0b9770 msgr2=0x7f174c0b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.432+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 shutdown_connections 2020-11-20T13:56:33.434 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.433+0000 7f1751a6e700 1 -- 172.21.15.117:0/3745275736 wait complete. 2020-11-20T13:56:33.436 INFO:teuthology.orchestra.run.smithi117.stderr:new fs with metadata pool 5 and data pool 6 2020-11-20T13:56:33.446 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:33.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.552+0000 7fcb25f85700 1 Processor -- start 2020-11-20T13:56:33.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.552+0000 7fcb25f85700 1 -- start start 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb25f85700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb201438e0 0x7fcb20143d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb25f85700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fcb200ae6f0 0x7fcb200aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb25f85700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2009b920 con 0x7fcb20098e30 2020-11-20T13:56:33.555 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb25f85700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb20144230 con 0x7fcb200ae6f0 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb25f85700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb200b5510 con 0x7fcb201438e0 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb201438e0 0x7fcb20143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1f7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fcb20098e30 0x7fcb2009b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60816/0 (socket says 172.21.15.117:60816) 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb201438e0 0x7fcb20143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41660/0 (socket says 172.21.15.117:41660) 2020-11-20T13:56:33.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1f7fe700 1 -- 172.21.15.117:0/1205535620 learned_addr learned my addr 172.21.15.117:0/1205535620 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:33.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1ffff700 1 --2- 172.21.15.117:0/1205535620 >> v2:172.21.15.85:3300/0 conn(0x7fcb200ae6f0 0x7fcb200aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1effd700 1 -- 172.21.15.117:0/1205535620 >> v2:172.21.15.85:3300/0 conn(0x7fcb200ae6f0 msgr2=0x7fcb200aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1effd700 1 --2- 172.21.15.117:0/1205535620 >> v2:172.21.15.85:3300/0 conn(0x7fcb200ae6f0 0x7fcb200aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1effd700 1 -- 172.21.15.117:0/1205535620 >> v1:172.21.15.117:6789/0 conn(0x7fcb20098e30 legacy=0x7fcb2009b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1effd700 1 -- 172.21.15.117:0/1205535620 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200b57b0 con 0x7fcb201438e0 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1e7fc700 1 -- 172.21.15.117:0/1205535620 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014751263 0 0) 0x7fcb2009b920 con 0x7fcb20098e30 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.553+0000 7fcb1effd700 1 --2- 172.21.15.117:0/1205535620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb20143d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fcb14002f30 tx=0x7fcb14006dc0).ready entity=mon.1 client_cookie=335e0ad578bb1a34 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb1e7fc700 1 -- 172.21.15.117:0/1205535620 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb140097b0 con 0x7fcb201438e0 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb1e7fc700 1 -- 172.21.15.117:0/1205535620 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb14009910 con 0x7fcb201438e0 2020-11-20T13:56:33.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb1e7fc700 1 -- 172.21.15.117:0/1205535620 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb14009c70 con 0x7fcb201438e0 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 -- 172.21.15.117:0/1205535620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 msgr2=0x7fcb20143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 --2- 172.21.15.117:0/1205535620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb20143d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fcb14002f30 tx=0x7fcb14006dc0).stop 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 -- 172.21.15.117:0/1205535620 shutdown_connections 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 --2- 172.21.15.117:0/1205535620 >> v2:172.21.15.85:3300/0 conn(0x7fcb200ae6f0 0x7fcb200aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 --2- 172.21.15.117:0/1205535620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb20143d40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 -- 172.21.15.117:0/1205535620 >> 172.21.15.117:0/1205535620 conn(0x7fcb2005b9f0 msgr2=0x7fcb2005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 -- 172.21.15.117:0/1205535620 shutdown_connections 2020-11-20T13:56:33.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.554+0000 7fcb25f85700 1 -- 172.21.15.117:0/1205535620 wait complete. 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.555+0000 7fcb25f85700 1 Processor -- start 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.555+0000 7fcb25f85700 1 -- start start 2020-11-20T13:56:33.565 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb25f85700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb20098e30 0x7fcb20142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb25f85700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb200ae6f0 0x7fcb20062790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb25f85700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb2013c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb25f85700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2009d260 con 0x7fcb200ae6f0 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb25f85700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200b2db0 con 0x7fcb201438e0 2020-11-20T13:56:33.566 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb25f85700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200b3450 con 0x7fcb20098e30 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb2013c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb2013c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41664/0 (socket says 172.21.15.117:41664) 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 -- 172.21.15.117:0/3795121676 learned_addr learned my addr 172.21.15.117:0/3795121676 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:33.567 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1effd700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb200ae6f0 0x7fcb20062790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1f7fe700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb20098e30 0x7fcb20142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 -- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb20098e30 msgr2=0x7fcb20142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb20098e30 0x7fcb20142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 -- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb200ae6f0 msgr2=0x7fcb20062790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.568 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb200ae6f0 0x7fcb20062790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 -- 172.21.15.117:0/3795121676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200b20d0 con 0x7fcb201438e0 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1f7fe700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb20098e30 0x7fcb20142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1ffff700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb2013c2c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fcb18006dd0 tx=0x7fcb180091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.556+0000 7fcb1cff9700 1 -- 172.21.15.117:0/3795121676 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb1800ca00 con 0x7fcb201438e0 2020-11-20T13:56:33.569 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.557+0000 7fcb1cff9700 1 -- 172.21.15.117:0/3795121676 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1800cb60 con 0x7fcb201438e0 2020-11-20T13:56:33.570 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.557+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb200b23c0 con 0x7fcb201438e0 2020-11-20T13:56:33.570 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.557+0000 7fcb1cff9700 1 -- 172.21.15.117:0/3795121676 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb180123f0 con 0x7fcb201438e0 2020-11-20T13:56:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.557+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb20062e40 con 0x7fcb201438e0 2020-11-20T13:56:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.557+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb200b3610 con 0x7fcb201438e0 2020-11-20T13:56:33.571 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.558+0000 7fcb1cff9700 1 -- 172.21.15.117:0/3795121676 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fcb18022070 con 0x7fcb201438e0 2020-11-20T13:56:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.558+0000 7fcb1cff9700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fcb00051290 0x7fcb00053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.558+0000 7fcb1cff9700 1 -- 172.21.15.117:0/3795121676 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fcb180696a0 con 0x7fcb201438e0 2020-11-20T13:56:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.558+0000 7fcb1f7fe700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fcb00051290 0x7fcb00053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.558+0000 7fcb1f7fe700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fcb00051290 0x7fcb00053750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fcb1000b000 tx=0x7fcb10009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.561+0000 7fcb1cff9700 1 -- 172.21.15.117:0/3795121676 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fcb180418c0 con 0x7fcb201438e0 2020-11-20T13:56:33.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.711+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb20062e40 con 0x7fcb201438e0 2020-11-20T13:56:33.712 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.712+0000 7fcb1cff9700 1 -- 172.21.15.117:0/3795121676 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16332 (secure 0 0 0) 0x7fcb20062e40 con 0x7fcb201438e0 2020-11-20T13:56:33.713 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:33.713 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":34,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:56:33.416656+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:31.475882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:32.365985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:33.714 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fcb00051290 msgr2=0x7fcb00053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fcb00051290 0x7fcb00053750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fcb1000b000 tx=0x7fcb10009040).stop 2020-11-20T13:56:33.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 msgr2=0x7fcb2013c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb2013c2c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fcb18006dd0 tx=0x7fcb180091a0).stop 2020-11-20T13:56:33.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 shutdown_connections 2020-11-20T13:56:33.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fcb00051290 0x7fcb00053750 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb20098e30 0x7fcb20142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb201438e0 0x7fcb2013c2c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 --2- 172.21.15.117:0/3795121676 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb200ae6f0 0x7fcb20062790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.716 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 >> 172.21.15.117:0/3795121676 conn(0x7fcb2005b9f0 msgr2=0x7fcb200b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 shutdown_connections 2020-11-20T13:56:33.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.713+0000 7fcb25f85700 1 -- 172.21.15.117:0/3795121676 wait complete. 2020-11-20T13:56:33.727 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:33.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.834+0000 7f8f9d534700 1 Processor -- start 2020-11-20T13:56:33.834 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.834+0000 7f8f9d534700 1 -- start start 2020-11-20T13:56:33.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.834+0000 7f8f9d534700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 0x7f8f9809b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.834+0000 7f8f9d534700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8f980ae6f0 0x7f8f980aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.834+0000 7f8f9d534700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980b5510 con 0x7f8f9809b700 2020-11-20T13:56:33.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.834+0000 7f8f9d534700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f980b57b0 con 0x7f8f980ae6f0 2020-11-20T13:56:33.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.834+0000 7f8f9d534700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b5a50 con 0x7f8f98098e30 2020-11-20T13:56:33.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f8f9809b700 0x7f8f981438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60830/0 (socket says 172.21.15.117:60830) 2020-11-20T13:56:33.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 -- 172.21.15.117:0/1858302301 learned_addr learned my addr 172.21.15.117:0/1858302301 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:33.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f96ffd700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 0x7f8f9809b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277605386 0 0) 0x7f8f980b5510 con 0x7f8f9809b700 2020-11-20T13:56:33.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f977fe700 1 --2- 172.21.15.117:0/1858302301 >> v2:172.21.15.85:3300/0 conn(0x7f8f980ae6f0 0x7f8f980aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c003230 con 0x7f8f9809b700 2020-11-20T13:56:33.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3917436289 0 0) 0x7f8f7c003230 con 0x7f8f9809b700 2020-11-20T13:56:33.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 msgr2=0x7f8f9809b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 0x7f8f9809b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 >> v2:172.21.15.85:3300/0 conn(0x7f8f980ae6f0 msgr2=0x7f8f980aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 --2- 172.21.15.117:0/1858302301 >> v2:172.21.15.85:3300/0 conn(0x7f8f980ae6f0 0x7f8f980aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f98099210 con 0x7f8f9809b700 2020-11-20T13:56:33.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f8f8c0016d0 con 0x7f8f9809b700 2020-11-20T13:56:33.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 >> v1:172.21.15.117:6789/0 conn(0x7f8f9809b700 legacy=0x7f8f981438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f7c003720 0x7f8f7c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7c0044d0 0x7f8f7c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 0x7f8f7c004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.848 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b57b0 con 0x7f8f7c003720 2020-11-20T13:56:33.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b5a50 con 0x7f8f7c0044d0 2020-11-20T13:56:33.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c004260 con 0x7f8f98098e30 2020-11-20T13:56:33.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7c0044d0 0x7f8f7c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f96ffd700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f7c003720 0x7f8f7c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.849 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f977fe700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 0x7f8f7c004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 -- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 msgr2=0x7f8f7c004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98098e30 0x7f8f7c004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 -- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f7c003720 msgr2=0x7f8f7c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:33.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f7c003720 0x7f8f7c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.835+0000 7f8f967fc700 1 -- 172.21.15.117:0/1858302301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c003230 con 0x7f8f7c0044d0 2020-11-20T13:56:33.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f967fc700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7c0044d0 0x7f8f7c004950 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8f8c0013e0 tx=0x7f8f8c0077c0).ready entity=mon.1 client_cookie=5fff4191de581345 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8c00ce00 con 0x7f8f7c0044d0 2020-11-20T13:56:33.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0072e0 con 0x7f8f7c0044d0 2020-11-20T13:56:33.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f95ffb700 1 -- 172.21.15.117:0/1858302301 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8c0057d0 con 0x7f8f7c0044d0 2020-11-20T13:56:33.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f9d534700 1 -- 172.21.15.117:0/1858302301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7c0044d0 msgr2=0x7f8f7c004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f9d534700 1 --2- 172.21.15.117:0/1858302301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f7c0044d0 0x7f8f7c004950 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f8f8c0013e0 tx=0x7f8f8c0077c0).stop 2020-11-20T13:56:33.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f977fe700 1 -- 172.21.15.117:0/1858302301 reap_dead start 2020-11-20T13:56:33.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f9d534700 1 -- 172.21.15.117:0/1858302301 shutdown_connections 2020-11-20T13:56:33.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f9d534700 1 -- 172.21.15.117:0/1858302301 >> 172.21.15.117:0/1858302301 conn(0x7f8f9805b9f0 msgr2=0x7f8f9805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:33.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f9d534700 1 -- 172.21.15.117:0/1858302301 shutdown_connections 2020-11-20T13:56:33.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.836+0000 7f8f9d534700 1 -- 172.21.15.117:0/1858302301 wait complete. 2020-11-20T13:56:33.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.837+0000 7f8f9d534700 1 Processor -- start 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.837+0000 7f8f9d534700 1 -- start start 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f9d534700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980670f0 0x7f8f98063090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f9d534700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98156e40 0x7f8f98144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f9d534700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f981438d0 0x7f8f98145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f9d534700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98141550 con 0x7f8f981438d0 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f9d534700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98062950 con 0x7f8f980670f0 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f9d534700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98062bf0 con 0x7f8f98156e40 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980670f0 0x7f8f98063090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980670f0 0x7f8f98063090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41682/0 (socket says 172.21.15.117:41682) 2020-11-20T13:56:33.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 -- 172.21.15.117:0/2645118252 learned_addr learned my addr 172.21.15.117:0/2645118252 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f967fc700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98156e40 0x7f8f98144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f977fe700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f981438d0 0x7f8f98145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 -- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98156e40 msgr2=0x7f8f98144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98156e40 0x7f8f98144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.858 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 -- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f981438d0 msgr2=0x7f8f98145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f981438d0 0x7f8f98145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 -- 172.21.15.117:0/2645118252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980674d0 con 0x7f8f980670f0 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.838+0000 7f8f96ffd700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980670f0 0x7f8f98063090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f8f90006ad0 tx=0x7f8f900091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.839+0000 7f8f87fff700 1 -- 172.21.15.117:0/2645118252 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f9000fb90 con 0x7f8f980670f0 2020-11-20T13:56:33.859 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.839+0000 7f8f87fff700 1 -- 172.21.15.117:0/2645118252 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9000fcf0 con 0x7f8f980670f0 2020-11-20T13:56:33.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.839+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f98146190 con 0x7f8f980670f0 2020-11-20T13:56:33.860 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.839+0000 7f8f87fff700 1 -- 172.21.15.117:0/2645118252 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f90014630 con 0x7f8f980670f0 2020-11-20T13:56:33.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.839+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f980637c0 con 0x7f8f980670f0 2020-11-20T13:56:33.861 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.839+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f98057050 con 0x7f8f980670f0 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.840+0000 7f8f87fff700 1 -- 172.21.15.117:0/2645118252 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8f90024070 con 0x7f8f980670f0 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.840+0000 7f8f87fff700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8f740513c0 0x7f8f74053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.840+0000 7f8f87fff700 1 -- 172.21.15.117:0/2645118252 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f8f90069aa0 con 0x7f8f980670f0 2020-11-20T13:56:33.863 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.840+0000 7f8f967fc700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8f740513c0 0x7f8f74053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:33.864 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.840+0000 7f8f967fc700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8f740513c0 0x7f8f74053880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8f8c019000 tx=0x7f8f8c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:33.864 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.843+0000 7f8f87fff700 1 -- 172.21.15.117:0/2645118252 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8f90041cc0 con 0x7f8f980670f0 2020-11-20T13:56:33.996 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.995+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f9805ca00 con 0x7f8f980670f0 2020-11-20T13:56:33.997 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.996+0000 7f8f87fff700 1 -- 172.21.15.117:0/2645118252 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16332 (secure 0 0 0) 0x7f8f900452e0 con 0x7f8f980670f0 2020-11-20T13:56:33.997 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:33.997 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":34,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:56:33.416656+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:31.475882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:32.365985+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:33.998 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8f740513c0 msgr2=0x7f8f74053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8f740513c0 0x7f8f74053880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f8f8c019000 tx=0x7f8f8c015000).stop 2020-11-20T13:56:33.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980670f0 msgr2=0x7f8f98063090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:33.999 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980670f0 0x7f8f98063090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f8f90006ad0 tx=0x7f8f900091a0).stop 2020-11-20T13:56:34.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 shutdown_connections 2020-11-20T13:56:34.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8f740513c0 0x7f8f74053880 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8f981438d0 0x7f8f98145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.000 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980670f0 0x7f8f98063090 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 --2- 172.21.15.117:0/2645118252 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8f98156e40 0x7f8f98144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 >> 172.21.15.117:0/2645118252 conn(0x7f8f9805b9f0 msgr2=0x7f8f980b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 shutdown_connections 2020-11-20T13:56:34.001 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:33.998+0000 7f8f9d534700 1 -- 172.21.15.117:0/2645118252 wait complete. 2020-11-20T13:56:34.010 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-11-20T13:56:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.119+0000 7fd2716a6700 1 Processor -- start 2020-11-20T13:56:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.119+0000 7fd2716a6700 1 -- start start 2020-11-20T13:56:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.120+0000 7fd2716a6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 0x7fd26c05b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.120+0000 7fd2716a6700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c0b06d0 con 0x7fd26c09f6c0 2020-11-20T13:56:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.120+0000 7fd2716a6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c1494a0 con 0x7fd26c148a00 2020-11-20T13:56:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.120+0000 7fd2716a6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0b0da0 con 0x7fd26c144d20 2020-11-20T13:56:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.120+0000 7fd26affd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd26c148a00 0x7fd26c144610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60848/0 (socket says 172.21.15.117:60848) 2020-11-20T13:56:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.120+0000 7fd26affd700 1 -- 172.21.15.117:0/4048699871 learned_addr learned my addr 172.21.15.117:0/4048699871 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 0x7fd26c05b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785183136 0 0) 0x7fd26c1494a0 con 0x7fd26c148a00 2020-11-20T13:56:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd250003230 con 0x7fd26c148a00 2020-11-20T13:56:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 277661324 0 0) 0x7fd250003230 con 0x7fd26c148a00 2020-11-20T13:56:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 msgr2=0x7fd26c05b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 0x7fd26c05b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 >> v1:172.21.15.85:6789/0 conn(0x7fd26c09f6c0 legacy=0x7fd26c0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0b0900 con 0x7fd26c148a00 2020-11-20T13:56:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fd2600016d0 con 0x7fd26c148a00 2020-11-20T13:56:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 >> v1:172.21.15.117:6789/0 conn(0x7fd26c148a00 legacy=0x7fd26c144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd250003720 0x7fd250003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250004060 0x7fd250004530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 0x7fd250004bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0b0da0 con 0x7fd250003720 2020-11-20T13:56:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250005100 con 0x7fd250004060 2020-11-20T13:56:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2500052c0 con 0x7fd26c144d20 2020-11-20T13:56:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250004060 0x7fd250004530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26affd700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd250003720 0x7fd250003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26b7fe700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 0x7fd250004bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 -- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 msgr2=0x7fd250004bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c144d20 0x7fd250004bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 -- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd250003720 msgr2=0x7fd250003b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd250003720 0x7fd250003b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 -- 172.21.15.117:0/4048699871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250003230 con 0x7fd250004060 2020-11-20T13:56:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250004060 0x7fd250004530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fd25c00c000 tx=0x7fd25c008040).ready entity=mon.1 client_cookie=5c35e4183b4dddc4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.121+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd25c00e8a0 con 0x7fd250004060 2020-11-20T13:56:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c00ea00 con 0x7fd250004060 2020-11-20T13:56:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd269ffb700 1 -- 172.21.15.117:0/4048699871 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd25c00ed60 con 0x7fd250004060 2020-11-20T13:56:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd2716a6700 1 -- 172.21.15.117:0/4048699871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250004060 msgr2=0x7fd250004530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.143 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd2716a6700 1 --2- 172.21.15.117:0/4048699871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd250004060 0x7fd250004530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fd25c00c000 tx=0x7fd25c008040).stop 2020-11-20T13:56:34.143 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd26b7fe700 1 -- 172.21.15.117:0/4048699871 reap_dead start 2020-11-20T13:56:34.143 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd2716a6700 1 -- 172.21.15.117:0/4048699871 shutdown_connections 2020-11-20T13:56:34.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd2716a6700 1 -- 172.21.15.117:0/4048699871 >> 172.21.15.117:0/4048699871 conn(0x7fd26c0b6970 msgr2=0x7fd26c0b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd2716a6700 1 -- 172.21.15.117:0/4048699871 shutdown_connections 2020-11-20T13:56:34.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.122+0000 7fd2716a6700 1 -- 172.21.15.117:0/4048699871 wait complete. 2020-11-20T13:56:34.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.123+0000 7fd2716a6700 1 Processor -- start 2020-11-20T13:56:34.144 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.123+0000 7fd2716a6700 1 -- start start 2020-11-20T13:56:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd26c0b38b0 0x7fd26c145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c132100 0x7fd26c132580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c062e30 0x7fd26c063290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c132ff0 con 0x7fd26c0b38b0 2020-11-20T13:56:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c133290 con 0x7fd26c132100 2020-11-20T13:56:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c133530 con 0x7fd26c062e30 2020-11-20T13:56:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c132100 0x7fd26c132580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c132100 0x7fd26c132580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41700/0 (socket says 172.21.15.117:41700) 2020-11-20T13:56:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 -- 172.21.15.117:0/2022696049 learned_addr learned my addr 172.21.15.117:0/2022696049 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26affd700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd26c0b38b0 0x7fd26c145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26b7fe700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c062e30 0x7fd26c063290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 -- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c062e30 msgr2=0x7fd26c063290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c062e30 0x7fd26c063290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 -- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd26c0b38b0 msgr2=0x7fd26c145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd26c0b38b0 0x7fd26c145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 -- 172.21.15.117:0/2022696049 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0b3c90 con 0x7fd26c132100 2020-11-20T13:56:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd26a7fc700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c132100 0x7fd26c132580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fd25c00c000 tx=0x7fd25c01a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd25bfff700 1 -- 172.21.15.117:0/2022696049 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd25c01c620 con 0x7fd26c132100 2020-11-20T13:56:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd25bfff700 1 -- 172.21.15.117:0/2022696049 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c01c780 con 0x7fd26c132100 2020-11-20T13:56:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c132a70 con 0x7fd26c132100 2020-11-20T13:56:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd25bfff700 1 -- 172.21.15.117:0/2022696049 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd25c01cae0 con 0x7fd26c132100 2020-11-20T13:56:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.124+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c146000 con 0x7fd26c132100 2020-11-20T13:56:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.125+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c1336f0 con 0x7fd26c132100 2020-11-20T13:56:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.125+0000 7fd25bfff700 1 -- 172.21.15.117:0/2022696049 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd25c02c070 con 0x7fd26c132100 2020-11-20T13:56:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.126+0000 7fd25bfff700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd248051210 0x7fd2480536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.126+0000 7fd25bfff700 1 -- 172.21.15.117:0/2022696049 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd25c06ac90 con 0x7fd26c132100 2020-11-20T13:56:34.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.126+0000 7fd26affd700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd248051210 0x7fd2480536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.151 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.126+0000 7fd26affd700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd248051210 0x7fd2480536d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fd260005000 tx=0x7fd260001e50).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.152 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.129+0000 7fd25bfff700 1 -- 172.21.15.117:0/2022696049 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd25c042eb0 con 0x7fd26c132100 2020-11-20T13:56:34.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.331+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd26c057050 con 0x7fd26c132100 2020-11-20T13:56:34.421 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.421+0000 7fd25bfff700 1 -- 172.21.15.117:0/2022696049 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v10) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd25c016030 con 0x7fd26c132100 2020-11-20T13:56:34.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd248051210 msgr2=0x7fd2480536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd248051210 0x7fd2480536d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fd260005000 tx=0x7fd260001e50).stop 2020-11-20T13:56:34.423 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c132100 msgr2=0x7fd26c132580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c132100 0x7fd26c132580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fd25c00c000 tx=0x7fd25c01a000).stop 2020-11-20T13:56:34.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 shutdown_connections 2020-11-20T13:56:34.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd248051210 0x7fd2480536d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.424 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd26c062e30 0x7fd26c063290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd26c0b38b0 0x7fd26c145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 --2- 172.21.15.117:0/2022696049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd26c132100 0x7fd26c132580 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 >> 172.21.15.117:0/2022696049 conn(0x7fd26c0b6970 msgr2=0x7fd26c0af780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.425 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 shutdown_connections 2020-11-20T13:56:34.426 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.422+0000 7fd2716a6700 1 -- 172.21.15.117:0/2022696049 wait complete. 2020-11-20T13:56:34.433 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:34.540 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.539+0000 7f45b0cfc700 1 Processor -- start 2020-11-20T13:56:34.540 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.539+0000 7f45b0cfc700 1 -- start start 2020-11-20T13:56:34.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45b0cfc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45ac098e30 0x7f45ac09b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45b0cfc700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f45ac09b700 0x7f45ac1438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45b0cfc700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0b5510 con 0x7f45ac143dd0 2020-11-20T13:56:34.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45b0cfc700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45ac0b57b0 con 0x7f45ac09b700 2020-11-20T13:56:34.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45b0cfc700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45ac0b5a50 con 0x7f45ac098e30 2020-11-20T13:56:34.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45aa59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45ac098e30 0x7f45ac09b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45aad9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f45ac143dd0 0x7f45ac0b60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60870/0 (socket says 172.21.15.117:60870) 2020-11-20T13:56:34.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.540+0000 7f45aad9d700 1 -- 172.21.15.117:0/1261597712 learned_addr learned my addr 172.21.15.117:0/1261597712 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:34.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45a9d9b700 1 --2- 172.21.15.117:0/1261597712 >> v2:172.21.15.85:3300/0 conn(0x7f45ac09b700 0x7f45ac1438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45aa59c700 1 -- 172.21.15.117:0/1261597712 >> v2:172.21.15.85:3300/0 conn(0x7f45ac09b700 msgr2=0x7f45ac1438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45aa59c700 1 --2- 172.21.15.117:0/1261597712 >> v2:172.21.15.85:3300/0 conn(0x7f45ac09b700 0x7f45ac1438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45aa59c700 1 -- 172.21.15.117:0/1261597712 >> v1:172.21.15.117:6789/0 conn(0x7f45ac143dd0 legacy=0x7f45ac0b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45a959a700 1 -- 172.21.15.117:0/1261597712 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554320032 0 0) 0x7f45ac0b5510 con 0x7f45ac143dd0 2020-11-20T13:56:34.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45aa59c700 1 -- 172.21.15.117:0/1261597712 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac099210 con 0x7f45ac098e30 2020-11-20T13:56:34.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45aa59c700 1 --2- 172.21.15.117:0/1261597712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac09b210 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f45a0002f30 tx=0x7f45a0006dc0).ready entity=mon.1 client_cookie=f616d59451e888fd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45a959a700 1 -- 172.21.15.117:0/1261597712 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a00097f0 con 0x7f45ac098e30 2020-11-20T13:56:34.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45a959a700 1 -- 172.21.15.117:0/1261597712 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a0009950 con 0x7f45ac098e30 2020-11-20T13:56:34.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45a959a700 1 -- 172.21.15.117:0/1261597712 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a0009cb0 con 0x7f45ac098e30 2020-11-20T13:56:34.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45b0cfc700 1 -- 172.21.15.117:0/1261597712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 msgr2=0x7f45ac09b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.541+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/1261597712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac09b210 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f45a0002f30 tx=0x7f45a0006dc0).stop 2020-11-20T13:56:34.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.542+0000 7f45b0cfc700 1 -- 172.21.15.117:0/1261597712 shutdown_connections 2020-11-20T13:56:34.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.542+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/1261597712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac09b210 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.542+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/1261597712 >> v2:172.21.15.85:3300/0 conn(0x7f45ac09b700 0x7f45ac1438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.542+0000 7f45b0cfc700 1 -- 172.21.15.117:0/1261597712 >> 172.21.15.117:0/1261597712 conn(0x7f45ac05b9f0 msgr2=0x7f45ac05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.542+0000 7f45b0cfc700 1 -- 172.21.15.117:0/1261597712 shutdown_connections 2020-11-20T13:56:34.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.542+0000 7f45b0cfc700 1 -- 172.21.15.117:0/1261597712 wait complete. 2020-11-20T13:56:34.549 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.542+0000 7f45b0cfc700 1 Processor -- start 2020-11-20T13:56:34.549 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45b0cfc700 1 -- start start 2020-11-20T13:56:34.551 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45b0cfc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45b0cfc700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45ac09b700 0x7f45ac1466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45b0cfc700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45ac143dd0 0x7f45ac140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45b0cfc700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0b31b0 con 0x7f45ac09b700 2020-11-20T13:56:34.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45b0cfc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0b0ec0 con 0x7f45ac098e30 2020-11-20T13:56:34.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45b0cfc700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0b1160 con 0x7f45ac143dd0 2020-11-20T13:56:34.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45aa59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45aa59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41716/0 (socket says 172.21.15.117:41716) 2020-11-20T13:56:34.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45aa59c700 1 -- 172.21.15.117:0/2468428898 learned_addr learned my addr 172.21.15.117:0/2468428898 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:34.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45a9d9b700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45ac09b700 0x7f45ac1466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45aad9d700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45ac143dd0 0x7f45ac140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45aa59c700 1 -- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45ac143dd0 msgr2=0x7f45ac140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.543+0000 7f45aa59c700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45ac143dd0 0x7f45ac140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f45aa59c700 1 -- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45ac09b700 msgr2=0x7f45ac1466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f45aa59c700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45ac09b700 0x7f45ac1466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f45aa59c700 1 -- 172.21.15.117:0/2468428898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac0b2910 con 0x7f45ac098e30 2020-11-20T13:56:34.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f45aad9d700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45ac143dd0 0x7f45ac140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:34.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f45aa59c700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac157fd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f45a0002f30 tx=0x7f45a0015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f459b7fe700 1 -- 172.21.15.117:0/2468428898 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a0011c30 con 0x7f45ac098e30 2020-11-20T13:56:34.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f459b7fe700 1 -- 172.21.15.117:0/2468428898 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a0011d90 con 0x7f45ac098e30 2020-11-20T13:56:34.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f459b7fe700 1 -- 172.21.15.117:0/2468428898 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45a001d640 con 0x7f45ac098e30 2020-11-20T13:56:34.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac0b2b60 con 0x7f45ac098e30 2020-11-20T13:56:34.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.544+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac154230 con 0x7f45ac098e30 2020-11-20T13:56:34.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.545+0000 7f459b7fe700 1 -- 172.21.15.117:0/2468428898 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f45a0024030 con 0x7f45ac098e30 2020-11-20T13:56:34.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.545+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac0b1320 con 0x7f45ac098e30 2020-11-20T13:56:34.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.545+0000 7f459b7fe700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.559 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.545+0000 7f459b7fe700 1 -- 172.21.15.117:0/2468428898 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f45a006a330 con 0x7f45ac098e30 2020-11-20T13:56:34.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.545+0000 7f45a9d9b700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.546+0000 7f45a9d9b700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f459c00c000 tx=0x7f459c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.560 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.549+0000 7f459b7fe700 1 -- 172.21.15.117:0/2468428898 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f45a0042550 con 0x7f45ac098e30 2020-11-20T13:56:34.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.748+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45ac057050 con 0x7f45ac098e30 2020-11-20T13:56:34.749 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.748+0000 7f459b7fe700 1 -- 172.21.15.117:0/2468428898 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f45a0045b70 con 0x7f45ac098e30 2020-11-20T13:56:34.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:34.750 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/332543540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":332543540},{"type":"v1","addr":"172.21.15.85:6833","nonce":332543540}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:34.417958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4996":{"gid":4996,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/2328953218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2328953218},{"type":"v1","addr":"172.21.15.117:6835","nonce":2328953218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:34.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 msgr2=0x7f45880536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f459c00c000 tx=0x7f459c008040).stop 2020-11-20T13:56:34.751 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 msgr2=0x7f45ac157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac157fd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f45a0002f30 tx=0x7f45a0015000).stop 2020-11-20T13:56:34.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 shutdown_connections 2020-11-20T13:56:34.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4588051210 0x7f45880536d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f45ac143dd0 0x7f45ac140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac098e30 0x7f45ac157fd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 --2- 172.21.15.117:0/2468428898 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f45ac09b700 0x7f45ac1466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 >> 172.21.15.117:0/2468428898 conn(0x7f45ac05b9f0 msgr2=0x7f45ac0b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 shutdown_connections 2020-11-20T13:56:34.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.750+0000 7f45b0cfc700 1 -- 172.21.15.117:0/2468428898 wait complete. 2020-11-20T13:56:34.754 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 10 2020-11-20T13:56:34.761 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:34.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.866+0000 7f886f594700 1 Processor -- start 2020-11-20T13:56:34.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.866+0000 7f886f594700 1 -- start start 2020-11-20T13:56:34.875 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f886f594700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 0x7f88680b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f886f594700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680afdf0 con 0x7f886805c490 2020-11-20T13:56:34.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f886f594700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680b0090 con 0x7f8868147260 2020-11-20T13:56:34.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f886f594700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88680afbe0 con 0x7f886805bd70 2020-11-20T13:56:34.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f886db31700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f8868147260 0x7f8868060130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60888/0 (socket says 172.21.15.117:60888) 2020-11-20T13:56:34.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f886db31700 1 -- 172.21.15.117:0/333415157 learned_addr learned my addr 172.21.15.117:0/333415157 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:34.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302309195 0 0) 0x7f88680b0090 con 0x7f8868147260 2020-11-20T13:56:34.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f886d330700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 0x7f88680b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.867+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8850003230 con 0x7f8868147260 2020-11-20T13:56:34.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2985702975 0 0) 0x7f8850003230 con 0x7f8868147260 2020-11-20T13:56:34.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 msgr2=0x7f88680b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 0x7f88680b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 >> v1:172.21.15.85:6789/0 conn(0x7f886805c490 legacy=0x7f8868146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886805c150 con 0x7f8868147260 2020-11-20T13:56:34.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f8860001a50 con 0x7f8868147260 2020-11-20T13:56:34.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 >> v1:172.21.15.117:6789/0 conn(0x7f8868147260 legacy=0x7f8868060130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8850003720 0x7f8850003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8850004060 0x7f8850004530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 0x7f8850004b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680afbe0 con 0x7f8850004060 2020-11-20T13:56:34.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88500050d0 con 0x7f8850003720 2020-11-20T13:56:34.881 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8850005290 con 0x7f886805bd70 2020-11-20T13:56:34.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886d330700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8850003720 0x7f8850003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886cb2f700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8850004060 0x7f8850004530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886db31700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 0x7f8850004b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.882 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886d330700 1 -- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 msgr2=0x7f8850004b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886d330700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f886805bd70 0x7f8850004b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886d330700 1 -- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8850004060 msgr2=0x7f8850004530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:34.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886d330700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8850004060 0x7f8850004530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886d330700 1 -- 172.21.15.117:0/333415157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850003230 con 0x7f8850003720 2020-11-20T13:56:34.883 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886d330700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8850003720 0x7f8850003b00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f885800c000 tx=0x7f8858008040).ready entity=mon.1 client_cookie=3983ebfa21e42435 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f885800e8a0 con 0x7f8850003720 2020-11-20T13:56:34.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885800ea00 con 0x7f8850003720 2020-11-20T13:56:34.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f8857fff700 1 -- 172.21.15.117:0/333415157 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f885800ed60 con 0x7f8850003720 2020-11-20T13:56:34.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886f594700 1 -- 172.21.15.117:0/333415157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8850003720 msgr2=0x7f8850003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.884 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886f594700 1 --2- 172.21.15.117:0/333415157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8850003720 0x7f8850003b00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f885800c000 tx=0x7f8858008040).stop 2020-11-20T13:56:34.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.868+0000 7f886db31700 1 -- 172.21.15.117:0/333415157 reap_dead start 2020-11-20T13:56:34.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.869+0000 7f886f594700 1 -- 172.21.15.117:0/333415157 shutdown_connections 2020-11-20T13:56:34.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.869+0000 7f886f594700 1 -- 172.21.15.117:0/333415157 >> 172.21.15.117:0/333415157 conn(0x7f88680ba8b0 msgr2=0x7f88680bacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:34.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.869+0000 7f886f594700 1 -- 172.21.15.117:0/333415157 shutdown_connections 2020-11-20T13:56:34.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.869+0000 7f886f594700 1 -- 172.21.15.117:0/333415157 wait complete. 2020-11-20T13:56:34.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.870+0000 7f886f594700 1 Processor -- start 2020-11-20T13:56:34.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.870+0000 7f886f594700 1 -- start start 2020-11-20T13:56:34.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.870+0000 7f886f594700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f88680b2ab0 0x7f88680b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.870+0000 7f886f594700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f886809a3b0 0x7f886809c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886f594700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8868157480 0x7f8868156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.887 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886f594700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868156140 con 0x7f88680b2ab0 2020-11-20T13:56:34.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886f594700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681563e0 con 0x7f886809a3b0 2020-11-20T13:56:34.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886f594700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886809c500 con 0x7f8868157480 2020-11-20T13:56:34.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f886809a3b0 0x7f886809c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f886809a3b0 0x7f886809c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41738/0 (socket says 172.21.15.117:41738) 2020-11-20T13:56:34.888 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 -- 172.21.15.117:0/3475663510 learned_addr learned my addr 172.21.15.117:0/3475663510 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:34.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886db31700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8868157480 0x7f8868156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886d330700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f88680b2ab0 0x7f88680b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 -- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8868157480 msgr2=0x7f8868156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.889 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8868157480 0x7f8868156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 -- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f88680b2ab0 msgr2=0x7f88680b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:34.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f88680b2ab0 0x7f88680b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:34.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 -- 172.21.15.117:0/3475663510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886809c7c0 con 0x7f886809a3b0 2020-11-20T13:56:34.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886d330700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f88680b2ab0 0x7f88680b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:34.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886cb2f700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f886809a3b0 0x7f886809c010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f886400d000 tx=0x7f8864007b40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f88567fc700 1 -- 172.21.15.117:0/3475663510 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f886400f9b0 con 0x7f886809a3b0 2020-11-20T13:56:34.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f88567fc700 1 -- 172.21.15.117:0/3475663510 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886400fb10 con 0x7f886809a3b0 2020-11-20T13:56:34.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88681458a0 con 0x7f886809a3b0 2020-11-20T13:56:34.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f88567fc700 1 -- 172.21.15.117:0/3475663510 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f886400fe70 con 0x7f886809a3b0 2020-11-20T13:56:34.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.871+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8868157ac0 con 0x7f886809a3b0 2020-11-20T13:56:34.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.872+0000 7f88567fc700 1 -- 172.21.15.117:0/3475663510 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f886401b030 con 0x7f886809a3b0 2020-11-20T13:56:34.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.872+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88681565a0 con 0x7f886809a3b0 2020-11-20T13:56:34.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.873+0000 7f88567fc700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8848051290 0x7f8848053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:34.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.873+0000 7f88567fc700 1 -- 172.21.15.117:0/3475663510 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f88640690b0 con 0x7f886809a3b0 2020-11-20T13:56:34.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.873+0000 7f886d330700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8848051290 0x7f8848053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:34.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.873+0000 7f886d330700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8848051290 0x7f8848053750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8858007000 tx=0x7f885801a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:34.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:34.876+0000 7f88567fc700 1 -- 172.21.15.117:0/3475663510 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f88640412d0 con 0x7f886809a3b0 2020-11-20T13:56:35.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.026+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8868057050 con 0x7f886809a3b0 2020-11-20T13:56:35.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.027+0000 7f88567fc700 1 -- 172.21.15.117:0/3475663510 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f88640448f0 con 0x7f886809a3b0 2020-11-20T13:56:35.028 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:35.028 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":35,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:56:34.418235+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:31.475882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:32.365985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:35.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.028+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8848051290 msgr2=0x7f8848053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.028+0000 7f886f594700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8848051290 0x7f8848053750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8858007000 tx=0x7f885801a000).stop 2020-11-20T13:56:35.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.028+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f886809a3b0 msgr2=0x7f886809c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.028+0000 7f886f594700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f886809a3b0 0x7f886809c010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f886400d000 tx=0x7f8864007b40).stop 2020-11-20T13:56:35.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 shutdown_connections 2020-11-20T13:56:35.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8848051290 0x7f8848053750 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8868157480 0x7f8868156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f88680b2ab0 0x7f88680b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 --2- 172.21.15.117:0/3475663510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f886809a3b0 0x7f886809c010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 >> 172.21.15.117:0/3475663510 conn(0x7f88680ba8b0 msgr2=0x7f88680b9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 shutdown_connections 2020-11-20T13:56:35.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.029+0000 7f886f594700 1 -- 172.21.15.117:0/3475663510 wait complete. 2020-11-20T13:56:35.039 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-20T13:56:35.155 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.152+0000 7ff7eed53700 1 Processor -- start 2020-11-20T13:56:35.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.152+0000 7ff7eed53700 1 -- start start 2020-11-20T13:56:35.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7eed53700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e805f2d0 0x7ff7e80b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7eed53700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7ff7e80ae930 0x7ff7e809b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7eed53700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e80af250 con 0x7ff7e80b4300 2020-11-20T13:56:35.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7eed53700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e80991f0 con 0x7ff7e80ae930 2020-11-20T13:56:35.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7eed53700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8098ab0 con 0x7ff7e805f2d0 2020-11-20T13:56:35.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7ed2f0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e805f2d0 0x7ff7e80b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7ecaef700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7ff7e80ae930 0x7ff7e809b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.157 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7edaf1700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7ff7e80b4300 0x7ff7e8099590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.117:44808/0 (socket says 172.21.15.117:44808) 2020-11-20T13:56:35.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.153+0000 7ff7edaf1700 1 -- 172.21.15.117:0/2735662892 learned_addr learned my addr 172.21.15.117:0/2735662892 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:35.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7ed2f0700 1 -- 172.21.15.117:0/2735662892 >> v2:172.21.15.85:3300/0 conn(0x7ff7e80ae930 msgr2=0x7ff7e809b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7ed2f0700 1 --2- 172.21.15.117:0/2735662892 >> v2:172.21.15.85:3300/0 conn(0x7ff7e80ae930 0x7ff7e809b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7ed2f0700 1 -- 172.21.15.117:0/2735662892 >> v1:172.21.15.85:6789/0 conn(0x7ff7e80b4300 legacy=0x7ff7e8099590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7ed2f0700 1 -- 172.21.15.117:0/2735662892 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8098d50 con 0x7ff7e805f2d0 2020-11-20T13:56:35.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7dffff700 1 -- 172.21.15.117:0/2735662892 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904585800 0 0) 0x7ff7e80af250 con 0x7ff7e80b4300 2020-11-20T13:56:35.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7ecaef700 1 --2- 172.21.15.117:0/2735662892 >> v2:172.21.15.85:3300/0 conn(0x7ff7e80ae930 0x7ff7e809b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T13:56:35.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7ed2f0700 1 --2- 172.21.15.117:0/2735662892 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e805f2d0 0x7ff7e80b3e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff7e40032b0 tx=0x7ff7e40091a0).ready entity=mon.2 client_cookie=2cc1314a4c11ce98 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7dffff700 1 -- 172.21.15.117:0/2735662892 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e400b9a0 con 0x7ff7e805f2d0 2020-11-20T13:56:35.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7dffff700 1 -- 172.21.15.117:0/2735662892 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400bb00 con 0x7ff7e805f2d0 2020-11-20T13:56:35.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7dffff700 1 -- 172.21.15.117:0/2735662892 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e400bc70 con 0x7ff7e805f2d0 2020-11-20T13:56:35.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 -- 172.21.15.117:0/2735662892 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e805f2d0 msgr2=0x7ff7e80b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 --2- 172.21.15.117:0/2735662892 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e805f2d0 0x7ff7e80b3e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff7e40032b0 tx=0x7ff7e40091a0).stop 2020-11-20T13:56:35.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 -- 172.21.15.117:0/2735662892 shutdown_connections 2020-11-20T13:56:35.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 --2- 172.21.15.117:0/2735662892 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e805f2d0 0x7ff7e80b3e10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 --2- 172.21.15.117:0/2735662892 >> v2:172.21.15.85:3300/0 conn(0x7ff7e80ae930 0x7ff7e809b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 -- 172.21.15.117:0/2735662892 >> 172.21.15.117:0/2735662892 conn(0x7ff7e80b9470 msgr2=0x7ff7e80b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 -- 172.21.15.117:0/2735662892 shutdown_connections 2020-11-20T13:56:35.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.154+0000 7ff7eed53700 1 -- 172.21.15.117:0/2735662892 wait complete. 2020-11-20T13:56:35.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 Processor -- start 2020-11-20T13:56:35.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 -- start start 2020-11-20T13:56:35.164 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e805f2d0 0x7ff7e80b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e80ae930 0x7ff7e8063580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff7e80b4300 0x7ff7e81463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8063a70 con 0x7ff7e80b4300 2020-11-20T13:56:35.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8062900 con 0x7ff7e805f2d0 2020-11-20T13:56:35.165 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7eed53700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8062ba0 con 0x7ff7e80ae930 2020-11-20T13:56:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7ed2f0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e805f2d0 0x7ff7e80b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7ed2f0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e805f2d0 0x7ff7e80b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41750/0 (socket says 172.21.15.117:41750) 2020-11-20T13:56:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.156+0000 7ff7ed2f0700 1 -- 172.21.15.117:0/1416996516 learned_addr learned my addr 172.21.15.117:0/1416996516 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7edaf1700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff7e80b4300 0x7ff7e81463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.166 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ecaef700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e80ae930 0x7ff7e8063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ed2f0700 1 -- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e80ae930 msgr2=0x7ff7e8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ed2f0700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e80ae930 0x7ff7e8063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ed2f0700 1 -- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff7e80b4300 msgr2=0x7ff7e81463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ed2f0700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff7e80b4300 0x7ff7e81463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.167 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ed2f0700 1 -- 172.21.15.117:0/1416996516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8062e40 con 0x7ff7e805f2d0 2020-11-20T13:56:35.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ed2f0700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e805f2d0 0x7ff7e80b76f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ff7e40032b0 tx=0x7ff7e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ddffb700 1 -- 172.21.15.117:0/1416996516 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e4011e80 con 0x7ff7e805f2d0 2020-11-20T13:56:35.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ddffb700 1 -- 172.21.15.117:0/1416996516 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e401f3f0 con 0x7ff7e805f2d0 2020-11-20T13:56:35.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e80630e0 con 0x7ff7e805f2d0 2020-11-20T13:56:35.168 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7ddffb700 1 -- 172.21.15.117:0/1416996516 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7e401f750 con 0x7ff7e805f2d0 2020-11-20T13:56:35.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.157+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e8137240 con 0x7ff7e805f2d0 2020-11-20T13:56:35.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.158+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e8057050 con 0x7ff7e805f2d0 2020-11-20T13:56:35.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.158+0000 7ff7ddffb700 1 -- 172.21.15.117:0/1416996516 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7ff7e4026030 con 0x7ff7e805f2d0 2020-11-20T13:56:35.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.158+0000 7ff7ddffb700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff7c80513b0 0x7ff7c8053870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.158+0000 7ff7ddffb700 1 -- 172.21.15.117:0/1416996516 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7ff7e4069990 con 0x7ff7e805f2d0 2020-11-20T13:56:35.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.158+0000 7ff7ecaef700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff7c80513b0 0x7ff7c8053870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.159+0000 7ff7ecaef700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff7c80513b0 0x7ff7c8053870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff7d800b000 tx=0x7ff7d8006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.170 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.162+0000 7ff7ddffb700 1 -- 172.21.15.117:0/1416996516 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7ff7e4041bb0 con 0x7ff7e805f2d0 2020-11-20T13:56:35.367 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.367+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ff7e8067c70 con 0x7ff7e805f2d0 2020-11-20T13:56:35.371 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.371+0000 7ff7ddffb700 1 -- 172.21.15.117:0/1416996516 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v11) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff7e40150e0 con 0x7ff7e805f2d0 2020-11-20T13:56:35.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.372+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff7c80513b0 msgr2=0x7ff7c8053870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff7c80513b0 0x7ff7c8053870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff7d800b000 tx=0x7ff7d8006dc0).stop 2020-11-20T13:56:35.373 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e805f2d0 msgr2=0x7ff7e80b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e805f2d0 0x7ff7e80b76f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ff7e40032b0 tx=0x7ff7e4016000).stop 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 shutdown_connections 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7ff7c80513b0 0x7ff7c8053870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.374 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7ff7e80b4300 0x7ff7e81463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7e805f2d0 0x7ff7e80b76f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 --2- 172.21.15.117:0/1416996516 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7ff7e80ae930 0x7ff7e8063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 >> 172.21.15.117:0/1416996516 conn(0x7ff7e80b9470 msgr2=0x7ff7e809a920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.375 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 shutdown_connections 2020-11-20T13:56:35.376 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.373+0000 7ff7eed53700 1 -- 172.21.15.117:0/1416996516 wait complete. 2020-11-20T13:56:35.376 INFO:teuthology.orchestra.run.smithi117.stderr:updated caps for client.0 2020-11-20T13:56:35.384 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-20T13:56:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.497+0000 7f6e30ea0700 1 Processor -- start 2020-11-20T13:56:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.497+0000 7f6e30ea0700 1 -- start start 2020-11-20T13:56:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e30ea0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e2c05f2d0 0x7f6e2c0b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e30ea0700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f6e2c0b4300 0x7f6e2c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e30ea0700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0af250 con 0x7f6e2c0ae930 2020-11-20T13:56:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e30ea0700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2c0991f0 con 0x7f6e2c0b4300 2020-11-20T13:56:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e30ea0700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2c098ab0 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e2c05f2d0 0x7f6e2c0b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e2c05f2d0 0x7f6e2c0b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41758/0 (socket says 172.21.15.117:41758) 2020-11-20T13:56:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2a59c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6e2c0ae930 0x7f6e2c09b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60918/0 (socket says 172.21.15.117:60918) 2020-11-20T13:56:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/498155077 learned_addr learned my addr 172.21.15.117:0/498155077 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/498155077 >> v2:172.21.15.85:3300/0 conn(0x7f6e2c0b4300 msgr2=0x7f6e2c15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2b59e700 1 --2- 172.21.15.117:0/498155077 >> v2:172.21.15.85:3300/0 conn(0x7f6e2c0b4300 0x7f6e2c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 --2- 172.21.15.117:0/498155077 >> v2:172.21.15.85:3300/0 conn(0x7f6e2c0b4300 0x7f6e2c15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/498155077 >> v1:172.21.15.117:6789/0 conn(0x7f6e2c0ae930 legacy=0x7f6e2c09b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e29d9b700 1 -- 172.21.15.117:0/498155077 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672357652 0 0) 0x7f6e2c0af250 con 0x7f6e2c0ae930 2020-11-20T13:56:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/498155077 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c098d50 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e2ad9d700 1 --2- 172.21.15.117:0/498155077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c0b3e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f6e1c0032b0 tx=0x7f6e1c009270).ready entity=mon.1 client_cookie=9a11a5cf90f5860f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e29d9b700 1 -- 172.21.15.117:0/498155077 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e1c00bc00 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e29d9b700 1 -- 172.21.15.117:0/498155077 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c00bd60 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e29d9b700 1 -- 172.21.15.117:0/498155077 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e1c011400 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e30ea0700 1 -- 172.21.15.117:0/498155077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 msgr2=0x7f6e2c0b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.498+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/498155077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c0b3e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f6e1c0032b0 tx=0x7f6e1c009270).stop 2020-11-20T13:56:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.499+0000 7f6e30ea0700 1 -- 172.21.15.117:0/498155077 shutdown_connections 2020-11-20T13:56:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.499+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/498155077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c0b3e10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.506 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.499+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/498155077 >> v2:172.21.15.85:3300/0 conn(0x7f6e2c0b4300 0x7f6e2c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.506 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.499+0000 7f6e30ea0700 1 -- 172.21.15.117:0/498155077 >> 172.21.15.117:0/498155077 conn(0x7f6e2c0b9470 msgr2=0x7f6e2c0b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.506 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.499+0000 7f6e30ea0700 1 -- 172.21.15.117:0/498155077 shutdown_connections 2020-11-20T13:56:35.506 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.499+0000 7f6e30ea0700 1 -- 172.21.15.117:0/498155077 wait complete. 2020-11-20T13:56:35.509 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.500+0000 7f6e30ea0700 1 Processor -- start 2020-11-20T13:56:35.509 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.500+0000 7f6e30ea0700 1 -- start start 2020-11-20T13:56:35.509 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e30ea0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.510 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e30ea0700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6e2c0ae930 0x7f6e2c0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e30ea0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6e2c0b4300 0x7f6e2c0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e30ea0700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c143a30 con 0x7f6e2c0ae930 2020-11-20T13:56:35.512 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e30ea0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c143cd0 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e30ea0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c143f70 con 0x7f6e2c0b4300 2020-11-20T13:56:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41762/0 (socket says 172.21.15.117:41762) 2020-11-20T13:56:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/1463832177 learned_addr learned my addr 172.21.15.117:0/1463832177 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:35.513 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2b59e700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6e2c0b4300 0x7f6e2c0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2a59c700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6e2c0ae930 0x7f6e2c0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6e2c0b4300 msgr2=0x7f6e2c0b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6e2c0b4300 0x7f6e2c0b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.514 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6e2c0ae930 msgr2=0x7f6e2c0b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6e2c0ae930 0x7f6e2c0b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 -- 172.21.15.117:0/1463832177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c062b00 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2a59c700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6e2c0ae930 0x7f6e2c0b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:35.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e2ad9d700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c147660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f6e1c0032b0 tx=0x7f6e1c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.515 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e1b7fe700 1 -- 172.21.15.117:0/1463832177 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e1c0107a0 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e1b7fe700 1 -- 172.21.15.117:0/1463832177 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c010900 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.501+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c144410 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.502+0000 7f6e1b7fe700 1 -- 172.21.15.117:0/1463832177 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e1c010c60 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.502+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c1437d0 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.516 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.503+0000 7f6e1b7fe700 1 -- 172.21.15.117:0/1463832177 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f6e1c026030 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.503+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c144130 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.503+0000 7f6e1b7fe700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6e10051350 0x7f6e10053810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.503+0000 7f6e1b7fe700 1 -- 172.21.15.117:0/1463832177 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6e1c06a2f0 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.517 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.503+0000 7f6e2a59c700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6e10051350 0x7f6e10053810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.503+0000 7f6e2a59c700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6e10051350 0x7f6e10053810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6e20008000 tx=0x7f6e20009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.518 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.506+0000 7f6e1b7fe700 1 -- 172.21.15.117:0/1463832177 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6e1c042510 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.711 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.711+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6e2c057050 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.715 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.715+0000 7f6e1b7fe700 1 -- 172.21.15.117:0/1463832177 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v12) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6e1c0150e0 con 0x7f6e2c05f2d0 2020-11-20T13:56:35.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6e10051350 msgr2=0x7f6e10053810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.717 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6e10051350 0x7f6e10053810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f6e20008000 tx=0x7f6e20009040).stop 2020-11-20T13:56:35.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 msgr2=0x7f6e2c147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c147660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f6e1c0032b0 tx=0x7f6e1c016000).stop 2020-11-20T13:56:35.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 shutdown_connections 2020-11-20T13:56:35.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6e10051350 0x7f6e10053810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.718 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6e2c0b4300 0x7f6e2c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c05f2d0 0x7f6e2c147660 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 --2- 172.21.15.117:0/1463832177 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6e2c0ae930 0x7f6e2c0b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 >> 172.21.15.117:0/1463832177 conn(0x7f6e2c0b9470 msgr2=0x7f6e2c0b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 shutdown_connections 2020-11-20T13:56:35.719 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.717+0000 7f6e30ea0700 1 -- 172.21.15.117:0/1463832177 wait complete. 2020-11-20T13:56:35.720 INFO:teuthology.orchestra.run.smithi117.stderr:updated caps for client.1 2020-11-20T13:56:35.727 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-20T13:56:35.841 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.838+0000 7f8853b9b700 1 Processor -- start 2020-11-20T13:56:35.841 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.838+0000 7f8853b9b700 1 -- start start 2020-11-20T13:56:35.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8853b9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f884c05fc60 0x7f884c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8853b9b700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c0af250 con 0x7f884c0ae930 2020-11-20T13:56:35.842 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8853b9b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c098ab0 con 0x7f884c05f2d0 2020-11-20T13:56:35.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8853b9b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c0991f0 con 0x7f884c05fc60 2020-11-20T13:56:35.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8852939700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f884c05fc60 0x7f884c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8852138700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f884c05f2d0 0x7f884c0b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60930/0 (socket says 172.21.15.117:60930) 2020-11-20T13:56:35.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8852138700 1 -- 172.21.15.117:0/3202128727 learned_addr learned my addr 172.21.15.117:0/3202128727 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:35.843 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8852939700 1 -- 172.21.15.117:0/3202128727 >> v1:172.21.15.117:6789/0 conn(0x7f884c05f2d0 legacy=0x7f884c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8852939700 1 -- 172.21.15.117:0/3202128727 >> v1:172.21.15.85:6789/0 conn(0x7f884c0ae930 legacy=0x7f884c09b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8851136700 1 -- 172.21.15.117:0/3202128727 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622374081 0 0) 0x7f884c098ab0 con 0x7f884c05f2d0 2020-11-20T13:56:35.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.839+0000 7f8852939700 1 -- 172.21.15.117:0/3202128727 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c098d50 con 0x7f884c05fc60 2020-11-20T13:56:35.844 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8852939700 1 --2- 172.21.15.117:0/3202128727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c05fc60 0x7f884c15f050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f88440032b0 tx=0x7f88440091a0).ready entity=mon.1 client_cookie=d801f39370385132 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8851136700 1 -- 172.21.15.117:0/3202128727 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884400ba50 con 0x7f884c05fc60 2020-11-20T13:56:35.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8851136700 1 -- 172.21.15.117:0/3202128727 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884400bbb0 con 0x7f884c05fc60 2020-11-20T13:56:35.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8851136700 1 -- 172.21.15.117:0/3202128727 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f884400bd20 con 0x7f884c05fc60 2020-11-20T13:56:35.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8853b9b700 1 -- 172.21.15.117:0/3202128727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c05fc60 msgr2=0x7f884c15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.845 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3202128727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c05fc60 0x7f884c15f050 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f88440032b0 tx=0x7f88440091a0).stop 2020-11-20T13:56:35.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8853b9b700 1 -- 172.21.15.117:0/3202128727 shutdown_connections 2020-11-20T13:56:35.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3202128727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c05fc60 0x7f884c15f050 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.846 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8853b9b700 1 -- 172.21.15.117:0/3202128727 >> 172.21.15.117:0/3202128727 conn(0x7f884c0b9470 msgr2=0x7f884c0b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:35.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8853b9b700 1 -- 172.21.15.117:0/3202128727 shutdown_connections 2020-11-20T13:56:35.847 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.840+0000 7f8853b9b700 1 -- 172.21.15.117:0/3202128727 wait complete. 2020-11-20T13:56:35.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.841+0000 7f8853b9b700 1 Processor -- start 2020-11-20T13:56:35.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.841+0000 7f8853b9b700 1 -- start start 2020-11-20T13:56:35.850 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8853b9b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f884c05f2d0 0x7f884c066bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8853b9b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f884c05fc60 0x7f884c1490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8853b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c0ae930 0x7f884c1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8853b9b700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c132fb0 con 0x7f884c05fc60 2020-11-20T13:56:35.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8853b9b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c132d10 con 0x7f884c0ae930 2020-11-20T13:56:35.851 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8853b9b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0b3050 con 0x7f884c05f2d0 2020-11-20T13:56:35.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c0ae930 0x7f884c1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c0ae930 0x7f884c1575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41780/0 (socket says 172.21.15.117:41780) 2020-11-20T13:56:35.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 -- 172.21.15.117:0/3070824355 learned_addr learned my addr 172.21.15.117:0/3070824355 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:35.852 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852138700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f884c05f2d0 0x7f884c066bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.853 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8851937700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f884c05fc60 0x7f884c1490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 -- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f884c05f2d0 msgr2=0x7f884c066bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f884c05f2d0 0x7f884c066bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 -- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f884c05fc60 msgr2=0x7f884c1490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:35.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f884c05fc60 0x7f884c1490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:35.854 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 -- 172.21.15.117:0/3070824355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c1495e0 con 0x7f884c0ae930 2020-11-20T13:56:35.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852138700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f884c05f2d0 0x7f884c066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:35.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8851937700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f884c05fc60 0x7f884c1490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:35.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.842+0000 7f8852939700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c0ae930 0x7f884c1575d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f88440032b0 tx=0x7f8844016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.843+0000 7f8842ffd700 1 -- 172.21.15.117:0/3070824355 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88440106e0 con 0x7f884c0ae930 2020-11-20T13:56:35.855 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.843+0000 7f8842ffd700 1 -- 172.21.15.117:0/3070824355 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8844010840 con 0x7f884c0ae930 2020-11-20T13:56:35.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.843+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c132710 con 0x7f884c0ae930 2020-11-20T13:56:35.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.843+0000 7f8842ffd700 1 -- 172.21.15.117:0/3070824355 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8844010ba0 con 0x7f884c0ae930 2020-11-20T13:56:35.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.843+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c0b0f20 con 0x7f884c0ae930 2020-11-20T13:56:35.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.844+0000 7f8842ffd700 1 -- 172.21.15.117:0/3070824355 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f8844026030 con 0x7f884c0ae930 2020-11-20T13:56:35.856 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.844+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c0b3210 con 0x7f884c0ae930 2020-11-20T13:56:35.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.844+0000 7f8842ffd700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8834051320 0x7f88340537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:35.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.844+0000 7f8842ffd700 1 -- 172.21.15.117:0/3070824355 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f884406a0c0 con 0x7f884c0ae930 2020-11-20T13:56:35.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.844+0000 7f8852138700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8834051320 0x7f88340537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:35.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.844+0000 7f8852138700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8834051320 0x7f88340537e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8848008000 tx=0x7f8848009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:35.857 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:35.847+0000 7f8842ffd700 1 -- 172.21.15.117:0/3070824355 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f8844042320 con 0x7f884c0ae930 2020-11-20T13:56:36.056 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.056+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f884c057050 con 0x7f884c0ae930 2020-11-20T13:56:36.059 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.059+0000 7f8842ffd700 1 -- 172.21.15.117:0/3070824355 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f88440240e0 con 0x7f884c0ae930 2020-11-20T13:56:36.062 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8834051320 msgr2=0x7f88340537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.062 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8834051320 0x7f88340537e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8848008000 tx=0x7f8848009040).stop 2020-11-20T13:56:36.062 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c0ae930 msgr2=0x7f884c1575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.062 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c0ae930 0x7f884c1575d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f88440032b0 tx=0x7f8844016000).stop 2020-11-20T13:56:36.063 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 shutdown_connections 2020-11-20T13:56:36.063 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8834051320 0x7f88340537e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.063 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f884c05f2d0 0x7f884c066bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.063 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f884c0ae930 0x7f884c1575d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.064 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 --2- 172.21.15.117:0/3070824355 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f884c05fc60 0x7f884c1490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.064 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 >> 172.21.15.117:0/3070824355 conn(0x7f884c0b9470 msgr2=0x7f884c0b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.064 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 shutdown_connections 2020-11-20T13:56:36.064 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.061+0000 7f8853b9b700 1 -- 172.21.15.117:0/3070824355 wait complete. 2020-11-20T13:56:36.066 INFO:teuthology.orchestra.run.smithi117.stderr:updated caps for client.2 2020-11-20T13:56:36.073 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.3 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2020-11-20T13:56:36.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.185+0000 7fd31db1f700 1 Processor -- start 2020-11-20T13:56:36.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.185+0000 7fd31db1f700 1 -- start start 2020-11-20T13:56:36.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31db1f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 0x7fd3180af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31db1f700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd318080b70 con 0x7fd318060580 2020-11-20T13:56:36.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31db1f700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd318060ea0 con 0x7fd31809a1e0 2020-11-20T13:56:36.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31db1f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3180610d0 con 0x7fd318025950 2020-11-20T13:56:36.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31cb1d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd31809a1e0 0x7fd3180b6270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60944/0 (socket says 172.21.15.117:60944) 2020-11-20T13:56:36.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31cb1d700 1 -- 172.21.15.117:0/3789343497 learned_addr learned my addr 172.21.15.117:0/3789343497 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:36.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31ed81700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 0x7fd3180af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037342469 0 0) 0x7fd318060ea0 con 0x7fd31809a1e0 2020-11-20T13:56:36.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd304003230 con 0x7fd31809a1e0 2020-11-20T13:56:36.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705643240 0 0) 0x7fd318080b70 con 0x7fd318060580 2020-11-20T13:56:36.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd318060ea0 con 0x7fd318060580 2020-11-20T13:56:36.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 335241970 0 0) 0x7fd304003230 con 0x7fd31809a1e0 2020-11-20T13:56:36.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 msgr2=0x7fd3180af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 0x7fd3180af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 >> v1:172.21.15.85:6789/0 conn(0x7fd318060580 legacy=0x7fd31809cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180afd40 con 0x7fd31809a1e0 2020-11-20T13:56:36.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7fd3140016d0 con 0x7fd31809a1e0 2020-11-20T13:56:36.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 >> v1:172.21.15.117:6789/0 conn(0x7fd31809a1e0 legacy=0x7fd3180b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3040038b0 0x7fd304003cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3040044d0 0x7fd304004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 0x7fd304004e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3180610d0 con 0x7fd3040038b0 2020-11-20T13:56:36.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304005330 con 0x7fd3040044d0 2020-11-20T13:56:36.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30404f730 con 0x7fd318025950 2020-11-20T13:56:36.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31cb1d700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3040044d0 0x7fd304004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31d31e700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3040038b0 0x7fd304003cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.186+0000 7fd31ed81700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 0x7fd304004e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31cb1d700 1 -- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 msgr2=0x7fd304004e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31cb1d700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318025950 0x7fd304004e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31cb1d700 1 -- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3040038b0 msgr2=0x7fd304003cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31cb1d700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd3040038b0 0x7fd304003cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31cb1d700 1 -- 172.21.15.117:0/3789343497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304003230 con 0x7fd3040044d0 2020-11-20T13:56:36.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31cb1d700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3040044d0 0x7fd304004950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd3040054a0 tx=0x7fd318063530).ready entity=mon.1 client_cookie=6645a46bb3463ab9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd31400ce70 con 0x7fd3040044d0 2020-11-20T13:56:36.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3140070d0 con 0x7fd3040044d0 2020-11-20T13:56:36.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd30ffff700 1 -- 172.21.15.117:0/3789343497 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd314007430 con 0x7fd3040044d0 2020-11-20T13:56:36.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31db1f700 1 -- 172.21.15.117:0/3789343497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3040044d0 msgr2=0x7fd304004950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.197 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31db1f700 1 --2- 172.21.15.117:0/3789343497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3040044d0 0x7fd304004950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd3040054a0 tx=0x7fd318063530).stop 2020-11-20T13:56:36.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31ed81700 1 -- 172.21.15.117:0/3789343497 reap_dead start 2020-11-20T13:56:36.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31db1f700 1 -- 172.21.15.117:0/3789343497 shutdown_connections 2020-11-20T13:56:36.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31db1f700 1 -- 172.21.15.117:0/3789343497 >> 172.21.15.117:0/3789343497 conn(0x7fd31805f880 msgr2=0x7fd3180ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31db1f700 1 -- 172.21.15.117:0/3789343497 shutdown_connections 2020-11-20T13:56:36.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.187+0000 7fd31db1f700 1 -- 172.21.15.117:0/3789343497 wait complete. 2020-11-20T13:56:36.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.188+0000 7fd31db1f700 1 Processor -- start 2020-11-20T13:56:36.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31db1f700 1 -- start start 2020-11-20T13:56:36.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31db1f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b65d0 0x7fd3180655f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31db1f700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318065ae0 0x7fd318148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31db1f700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd318148ff0 0x7fd318144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31db1f700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318148430 con 0x7fd318148ff0 2020-11-20T13:56:36.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31db1f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318065f70 con 0x7fd3180b65d0 2020-11-20T13:56:36.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31db1f700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd318148660 con 0x7fd318065ae0 2020-11-20T13:56:36.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31d31e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b65d0 0x7fd3180655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31d31e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b65d0 0x7fd3180655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41796/0 (socket says 172.21.15.117:41796) 2020-11-20T13:56:36.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.189+0000 7fd31d31e700 1 -- 172.21.15.117:0/1485034251 learned_addr learned my addr 172.21.15.117:0/1485034251 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:36.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31ed81700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd318148ff0 0x7fd318144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31cb1d700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318065ae0 0x7fd318148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31d31e700 1 -- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318065ae0 msgr2=0x7fd318148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31d31e700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318065ae0 0x7fd318148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31d31e700 1 -- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd318148ff0 msgr2=0x7fd318144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31d31e700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd318148ff0 0x7fd318144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31d31e700 1 -- 172.21.15.117:0/1485034251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180b69b0 con 0x7fd3180b65d0 2020-11-20T13:56:36.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31ed81700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd318148ff0 0x7fd318144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:36.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31d31e700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b65d0 0x7fd3180655f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fd30800d000 tx=0x7fd308007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.205 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd30dffb700 1 -- 172.21.15.117:0/1485034251 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd30800fc10 con 0x7fd3180b65d0 2020-11-20T13:56:36.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd30dffb700 1 -- 172.21.15.117:0/1485034251 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30800fd70 con 0x7fd3180b65d0 2020-11-20T13:56:36.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd318145320 con 0x7fd3180b65d0 2020-11-20T13:56:36.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd30dffb700 1 -- 172.21.15.117:0/1485034251 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd308014640 con 0x7fd3180b65d0 2020-11-20T13:56:36.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.190+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd318145790 con 0x7fd3180b65d0 2020-11-20T13:56:36.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.191+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd318057050 con 0x7fd3180b65d0 2020-11-20T13:56:36.206 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.191+0000 7fd30dffb700 1 -- 172.21.15.117:0/1485034251 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fd30801b030 con 0x7fd3180b65d0 2020-11-20T13:56:36.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.191+0000 7fd30dffb700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2f8051320 0x7fd2f80537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.191+0000 7fd30dffb700 1 -- 172.21.15.117:0/1485034251 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fd3080692a0 con 0x7fd3180b65d0 2020-11-20T13:56:36.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.191+0000 7fd31cb1d700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2f8051320 0x7fd2f80537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.192+0000 7fd31cb1d700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2f8051320 0x7fd2f80537e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd314017000 tx=0x7fd314013000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.207 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.194+0000 7fd30dffb700 1 -- 172.21.15.117:0/1485034251 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fd3080414c0 con 0x7fd3180b65d0 2020-11-20T13:56:36.400 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.400+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd3180620f0 con 0x7fd3180b65d0 2020-11-20T13:56:36.404 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.404+0000 7fd30dffb700 1 -- 172.21.15.117:0/1485034251 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.3", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.3 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd308044ae0 con 0x7fd3180b65d0 2020-11-20T13:56:36.406 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2f8051320 msgr2=0x7fd2f80537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.408 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2f8051320 0x7fd2f80537e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd314017000 tx=0x7fd314013000).stop 2020-11-20T13:56:36.409 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b65d0 msgr2=0x7fd3180655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.409 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b65d0 0x7fd3180655f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fd30800d000 tx=0x7fd308007ad0).stop 2020-11-20T13:56:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 shutdown_connections 2020-11-20T13:56:36.410 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fd2f8051320 0x7fd2f80537e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.411 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd318148ff0 0x7fd318144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b65d0 0x7fd3180655f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 --2- 172.21.15.117:0/1485034251 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd318065ae0 0x7fd318148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.412 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 >> 172.21.15.117:0/1485034251 conn(0x7fd31805f880 msgr2=0x7fd31809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.413 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 shutdown_connections 2020-11-20T13:56:36.414 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.406+0000 7fd31db1f700 1 -- 172.21.15.117:0/1485034251 wait complete. 2020-11-20T13:56:36.415 INFO:teuthology.orchestra.run.smithi117.stderr:updated caps for client.3 2020-11-20T13:56:36.416 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:36.523 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.523+0000 7f5a2b73d700 1 Processor -- start 2020-11-20T13:56:36.524 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.523+0000 7f5a2b73d700 1 -- start start 2020-11-20T13:56:36.532 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a2b73d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f5a240b0080 0x7f5a24146420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a2b73d700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a24098be0 con 0x7f5a2409c7a0 2020-11-20T13:56:36.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a2b73d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a24098e80 con 0x7f5a2409bfe0 2020-11-20T13:56:36.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a2b73d700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a24099120 con 0x7f5a240b0080 2020-11-20T13:56:36.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a28cd8700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f5a2409bfe0 0x7f5a24148a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:60964/0 (socket says 172.21.15.117:60964) 2020-11-20T13:56:36.533 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a28cd8700 1 -- 172.21.15.117:0/759138820 learned_addr learned my addr 172.21.15.117:0/759138820 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:36.534 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a294d9700 1 --2- 172.21.15.117:0/759138820 >> v2:172.21.15.85:3300/0 conn(0x7f5a240b0080 0x7f5a24146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.534 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819504312 0 0) 0x7f5a24098e80 con 0x7f5a2409bfe0 2020-11-20T13:56:36.534 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a0c003230 con 0x7f5a2409bfe0 2020-11-20T13:56:36.534 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3380835848 0 0) 0x7f5a0c003230 con 0x7f5a2409bfe0 2020-11-20T13:56:36.534 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 >> v2:172.21.15.85:3300/0 conn(0x7f5a240b0080 msgr2=0x7f5a24146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.535 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 --2- 172.21.15.117:0/759138820 >> v2:172.21.15.85:3300/0 conn(0x7f5a240b0080 0x7f5a24146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.535 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 >> v1:172.21.15.85:6789/0 conn(0x7f5a2409c7a0 legacy=0x7f5a24149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.535 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a240b0460 con 0x7f5a2409bfe0 2020-11-20T13:56:36.535 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f5a140016d0 con 0x7f5a2409bfe0 2020-11-20T13:56:36.535 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 >> v1:172.21.15.117:6789/0 conn(0x7f5a2409bfe0 legacy=0x7f5a24148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.536 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a0c003720 0x7f5a0c003b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.536 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c0044d0 0x7f5a0c004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.536 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a0c004e40 0x7f5a0c00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.536 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24099120 con 0x7f5a0c004e40 2020-11-20T13:56:36.536 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0042b0 con 0x7f5a0c003720 2020-11-20T13:56:36.537 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.524+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00bc10 con 0x7f5a0c0044d0 2020-11-20T13:56:36.537 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a294d9700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a0c003720 0x7f5a0c003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.537 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a29cda700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a0c004e40 0x7f5a0c00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.537 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a28cd8700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c0044d0 0x7f5a0c004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.537 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a294d9700 1 -- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c0044d0 msgr2=0x7f5a0c004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.538 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a294d9700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c0044d0 0x7f5a0c004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.538 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a294d9700 1 -- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a0c004e40 msgr2=0x7f5a0c00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:36.539 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a294d9700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a0c004e40 0x7f5a0c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.539 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a294d9700 1 -- 172.21.15.117:0/759138820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c003230 con 0x7f5a0c003720 2020-11-20T13:56:36.539 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a29cda700 1 -- 172.21.15.117:0/759138820 reap_dead start 2020-11-20T13:56:36.540 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a28cd8700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c0044d0 0x7f5a0c004950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:36.540 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a294d9700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a0c003720 0x7f5a0c003b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f5a2000c000 tx=0x7f5a20008040).ready entity=mon.1 client_cookie=a3e017cc8af73be6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2000e970 con 0x7f5a0c003720 2020-11-20T13:56:36.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2000ead0 con 0x7f5a0c003720 2020-11-20T13:56:36.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a13fff700 1 -- 172.21.15.117:0/759138820 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2000ee30 con 0x7f5a0c003720 2020-11-20T13:56:36.541 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a2b73d700 1 -- 172.21.15.117:0/759138820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a0c003720 msgr2=0x7f5a0c003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a0c003720 0x7f5a0c003b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f5a2000c000 tx=0x7f5a20008040).stop 2020-11-20T13:56:36.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a2b73d700 1 -- 172.21.15.117:0/759138820 shutdown_connections 2020-11-20T13:56:36.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.525+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/759138820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a0c003720 0x7f5a0c003b00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.542 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.526+0000 7f5a2b73d700 1 -- 172.21.15.117:0/759138820 >> 172.21.15.117:0/759138820 conn(0x7f5a2405b9f0 msgr2=0x7f5a2405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.526+0000 7f5a2b73d700 1 -- 172.21.15.117:0/759138820 shutdown_connections 2020-11-20T13:56:36.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.526+0000 7f5a2b73d700 1 -- 172.21.15.117:0/759138820 wait complete. 2020-11-20T13:56:36.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 Processor -- start 2020-11-20T13:56:36.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 -- start start 2020-11-20T13:56:36.543 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c003720 0x7f5a24136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a24143430 0x7f5a24141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a24141d10 0x7f5a240b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240b5340 con 0x7f5a24143430 2020-11-20T13:56:36.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240b64d0 con 0x7f5a24141d10 2020-11-20T13:56:36.544 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a2b73d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240b5530 con 0x7f5a0c003720 2020-11-20T13:56:36.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a29cda700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a24141d10 0x7f5a240b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a29cda700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a24141d10 0x7f5a240b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41818/0 (socket says 172.21.15.117:41818) 2020-11-20T13:56:36.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a29cda700 1 -- 172.21.15.117:0/1930224285 learned_addr learned my addr 172.21.15.117:0/1930224285 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:36.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a28cd8700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a24143430 0x7f5a24141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.545 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.527+0000 7f5a294d9700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c003720 0x7f5a24136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a29cda700 1 -- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c003720 msgr2=0x7f5a24136f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a29cda700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c003720 0x7f5a24136f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a29cda700 1 -- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a24143430 msgr2=0x7f5a24141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a29cda700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a24143430 0x7f5a24141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.546 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a29cda700 1 -- 172.21.15.117:0/1930224285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a240b1b70 con 0x7f5a24141d10 2020-11-20T13:56:36.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a29cda700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a24141d10 0x7f5a240b0820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5a240b1ef0 tx=0x7f5a1c007e30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a127fc700 1 -- 172.21.15.117:0/1930224285 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1c00c9a0 con 0x7f5a24141d10 2020-11-20T13:56:36.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a127fc700 1 -- 172.21.15.117:0/1930224285 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c00cb00 con 0x7f5a24141d10 2020-11-20T13:56:36.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a24153a70 con 0x7f5a24141d10 2020-11-20T13:56:36.547 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a127fc700 1 -- 172.21.15.117:0/1930224285 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a1c00ce60 con 0x7f5a24141d10 2020-11-20T13:56:36.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.528+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a240b4460 con 0x7f5a24141d10 2020-11-20T13:56:36.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.529+0000 7f5a127fc700 1 -- 172.21.15.117:0/1930224285 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5a1c01d460 con 0x7f5a24141d10 2020-11-20T13:56:36.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.529+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a240b1d00 con 0x7f5a24141d10 2020-11-20T13:56:36.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.529+0000 7f5a127fc700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5a040513c0 0x7f5a04053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:36.548 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.529+0000 7f5a127fc700 1 -- 172.21.15.117:0/1930224285 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f5a1c0694e0 con 0x7f5a24141d10 2020-11-20T13:56:36.549 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.529+0000 7f5a294d9700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5a040513c0 0x7f5a04053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:36.549 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.529+0000 7f5a294d9700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5a040513c0 0x7f5a04053880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5a20007000 tx=0x7f5a2001a000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:36.549 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.533+0000 7f5a127fc700 1 -- 172.21.15.117:0/1930224285 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5a1c0416a0 con 0x7f5a24141d10 2020-11-20T13:56:36.733 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.733+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a24057050 con 0x7f5a24141d10 2020-11-20T13:56:36.734 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.733+0000 7f5a127fc700 1 -- 172.21.15.117:0/1930224285 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f5a1c044cc0 con 0x7f5a24141d10 2020-11-20T13:56:36.734 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:36.734 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/332543540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":332543540},{"type":"v1","addr":"172.21.15.85:6833","nonce":332543540}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:34.417958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4996":{"gid":4996,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/2328953218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2328953218},{"type":"v1","addr":"172.21.15.117:6835","nonce":2328953218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:36.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.734+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5a040513c0 msgr2=0x7f5a04053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.735 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.734+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5a040513c0 0x7f5a04053880 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5a20007000 tx=0x7f5a2001a000).stop 2020-11-20T13:56:36.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.734+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a24141d10 msgr2=0x7f5a240b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:36.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.734+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a24141d10 0x7f5a240b0820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5a240b1ef0 tx=0x7f5a1c007e30).stop 2020-11-20T13:56:36.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 shutdown_connections 2020-11-20T13:56:36.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f5a040513c0 0x7f5a04053880 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.736 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a0c003720 0x7f5a24136f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a24141d10 0x7f5a240b0820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 --2- 172.21.15.117:0/1930224285 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a24143430 0x7f5a24141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 >> 172.21.15.117:0/1930224285 conn(0x7f5a2405b9f0 msgr2=0x7f5a240b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 shutdown_connections 2020-11-20T13:56:36.737 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:36.735+0000 7f5a2b73d700 1 -- 172.21.15.117:0/1930224285 wait complete. 2020-11-20T13:56:36.738 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 10 2020-11-20T13:56:36.947 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:33.416903+0000', 'modified': '2020-11-20T13:56:34.417958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4996': {'gid': 4996, 'name': 'a', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/2328953218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2328953218}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2328953218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:36.947 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:36.948 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:37.074 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":4996,"want_state":"up:active","state":"up:active","rank_uptime":3.647119038,"mdsmap_epoch":10,"osdmap_epoch":35,"osdmap_epoch_barrier":34,"uptime":10.447412146} 2020-11-20T13:56:37.075 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 4996, "mdsmap_epoch": 10, "osdmap_epoch": 35, "osdmap_epoch_barrier": 34, "rank_uptime": 3.647119038, "state": "up:active", "uptime": 10.447412146, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:37.076 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:37.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.185+0000 7f9175926700 1 Processor -- start 2020-11-20T13:56:37.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.185+0000 7f9175926700 1 -- start start 2020-11-20T13:56:37.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.186+0000 7f9175926700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f91700ae8b0 0x7f91700aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.186+0000 7f9175926700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700b5860 0x7f9170098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.186+0000 7f9175926700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91700b4e80 con 0x7f9170099080 2020-11-20T13:56:37.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.186+0000 7f9175926700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91700994f0 con 0x7f91700ae8b0 2020-11-20T13:56:37.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.186+0000 7f9175926700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91700b6900 con 0x7f91700b5860 2020-11-20T13:56:37.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f91700ae8b0 0x7f91700aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f91700ae8b0 0x7f91700aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41822/0 (socket says 172.21.15.117:41822) 2020-11-20T13:56:37.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 -- 172.21.15.117:0/3957129745 learned_addr learned my addr 172.21.15.117:0/3957129745 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:37.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916e7fc700 1 --2- 172.21.15.117:0/3957129745 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700b5860 0x7f9170098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 -- 172.21.15.117:0/3957129745 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700b5860 msgr2=0x7f9170098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 --2- 172.21.15.117:0/3957129745 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700b5860 0x7f9170098b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 -- 172.21.15.117:0/3957129745 >> v1:172.21.15.85:6789/0 conn(0x7f9170099080 legacy=0x7f91700b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 -- 172.21.15.117:0/3957129745 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700b5120 con 0x7f91700ae8b0 2020-11-20T13:56:37.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916e7fc700 1 --2- 172.21.15.117:0/3957129745 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700b5860 0x7f9170098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:37.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916effd700 1 --2- 172.21.15.117:0/3957129745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91700ae8b0 0x7f91700aec90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f91700b6a40 tx=0x7f9158006e80).ready entity=mon.1 client_cookie=d42d169a679d024a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916dffb700 1 -- 172.21.15.117:0/3957129745 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91580097b0 con 0x7f91700ae8b0 2020-11-20T13:56:37.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916dffb700 1 -- 172.21.15.117:0/3957129745 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9158009910 con 0x7f91700ae8b0 2020-11-20T13:56:37.192 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f916dffb700 1 -- 172.21.15.117:0/3957129745 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9158009c70 con 0x7f91700ae8b0 2020-11-20T13:56:37.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f9175926700 1 -- 172.21.15.117:0/3957129745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91700ae8b0 msgr2=0x7f91700aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.187+0000 7f9175926700 1 --2- 172.21.15.117:0/3957129745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91700ae8b0 0x7f91700aec90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f91700b6a40 tx=0x7f9158006e80).stop 2020-11-20T13:56:37.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.188+0000 7f9175926700 1 -- 172.21.15.117:0/3957129745 shutdown_connections 2020-11-20T13:56:37.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.188+0000 7f9175926700 1 --2- 172.21.15.117:0/3957129745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91700ae8b0 0x7f91700aec90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.188+0000 7f9175926700 1 --2- 172.21.15.117:0/3957129745 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700b5860 0x7f9170098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.188+0000 7f9175926700 1 -- 172.21.15.117:0/3957129745 >> 172.21.15.117:0/3957129745 conn(0x7f917005b9f0 msgr2=0x7f917005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.188+0000 7f9175926700 1 -- 172.21.15.117:0/3957129745 shutdown_connections 2020-11-20T13:56:37.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.188+0000 7f9175926700 1 -- 172.21.15.117:0/3957129745 wait complete. 2020-11-20T13:56:37.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 Processor -- start 2020-11-20T13:56:37.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 -- start start 2020-11-20T13:56:37.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9170099080 0x7f917009b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700ae8b0 0x7f917009f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.198 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91700b5860 0x7f9170066fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917009f660 con 0x7f91700b5860 2020-11-20T13:56:37.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91700674b0 con 0x7f9170099080 2020-11-20T13:56:37.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f9175926700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917009ea30 con 0x7f91700ae8b0 2020-11-20T13:56:37.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f916effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9170099080 0x7f917009b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.199 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f916effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9170099080 0x7f917009b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41828/0 (socket says 172.21.15.117:41828) 2020-11-20T13:56:37.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f916effd700 1 -- 172.21.15.117:0/1297498548 learned_addr learned my addr 172.21.15.117:0/1297498548 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:37.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f916e7fc700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700ae8b0 0x7f917009f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.189+0000 7f916f7fe700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91700b5860 0x7f9170066fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f916effd700 1 -- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700ae8b0 msgr2=0x7f917009f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.200 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f916effd700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700ae8b0 0x7f917009f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f916effd700 1 -- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91700b5860 msgr2=0x7f9170066fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f916effd700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91700b5860 0x7f9170066fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f916effd700 1 -- 172.21.15.117:0/1297498548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917009ecd0 con 0x7f9170099080 2020-11-20T13:56:37.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f916f7fe700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91700b5860 0x7f9170066fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:37.201 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f916effd700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9170099080 0x7f917009b950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f91700b6a40 tx=0x7f9158015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f9167fff700 1 -- 172.21.15.117:0/1297498548 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9158011bf0 con 0x7f9170099080 2020-11-20T13:56:37.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f9167fff700 1 -- 172.21.15.117:0/1297498548 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9158011d50 con 0x7f9170099080 2020-11-20T13:56:37.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917013f500 con 0x7f9170099080 2020-11-20T13:56:37.202 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f9167fff700 1 -- 172.21.15.117:0/1297498548 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f915801e650 con 0x7f9170099080 2020-11-20T13:56:37.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.190+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917013bbe0 con 0x7f9170099080 2020-11-20T13:56:37.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.191+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9170057050 con 0x7f9170099080 2020-11-20T13:56:37.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.191+0000 7f9167fff700 1 -- 172.21.15.117:0/1297498548 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9158025030 con 0x7f9170099080 2020-11-20T13:56:37.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.191+0000 7f9167fff700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9154051290 0x7f9154053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.203 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.191+0000 7f9167fff700 1 -- 172.21.15.117:0/1297498548 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f915806a510 con 0x7f9170099080 2020-11-20T13:56:37.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.191+0000 7f916e7fc700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9154051290 0x7f9154053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.192+0000 7f916e7fc700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9154051290 0x7f9154053750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f916000c000 tx=0x7f9160008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.204 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.195+0000 7f9167fff700 1 -- 172.21.15.117:0/1297498548 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9158042730 con 0x7f9170099080 2020-11-20T13:56:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.345+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f917009ee60 con 0x7f9170099080 2020-11-20T13:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.346+0000 7f9167fff700 1 -- 172.21.15.117:0/1297498548 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f9158023020 con 0x7f9170099080 2020-11-20T13:56:37.346 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:37.346 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":35,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:56:34.418235+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:31.475882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:32.365985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9154051290 msgr2=0x7f9154053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9154051290 0x7f9154053750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f916000c000 tx=0x7f9160008040).stop 2020-11-20T13:56:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9170099080 msgr2=0x7f917009b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9170099080 0x7f917009b950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f91700b6a40 tx=0x7f9158015000).stop 2020-11-20T13:56:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 shutdown_connections 2020-11-20T13:56:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9154051290 0x7f9154053750 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f91700b5860 0x7f9170066fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9170099080 0x7f917009b950 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 --2- 172.21.15.117:0/1297498548 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f91700ae8b0 0x7f917009f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 >> 172.21.15.117:0/1297498548 conn(0x7f917005b9f0 msgr2=0x7f91700b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 shutdown_connections 2020-11-20T13:56:37.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.347+0000 7f9175926700 1 -- 172.21.15.117:0/1297498548 wait complete. 2020-11-20T13:56:37.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:31.475882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:32.365985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:37.359 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:37.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.468+0000 7f250959b700 1 Processor -- start 2020-11-20T13:56:37.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.468+0000 7f250959b700 1 -- start start 2020-11-20T13:56:37.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.469+0000 7f250959b700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f250409bd80 0x7f250409cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.469+0000 7f250959b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2504143eb0 0x7f25040b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.469+0000 7f250959b700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25040999d0 con 0x7f250409a110 2020-11-20T13:56:37.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.469+0000 7f250959b700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2504099c70 con 0x7f250409bd80 2020-11-20T13:56:37.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.469+0000 7f250959b700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250409c160 con 0x7f2504143eb0 2020-11-20T13:56:37.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.469+0000 7f25037fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2504143eb0 0x7f25040b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25037fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2504143eb0 0x7f25040b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41836/0 (socket says 172.21.15.117:41836) 2020-11-20T13:56:37.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25027fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f250409a110 0x7f25041437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32768/0 (socket says 172.21.15.117:32768) 2020-11-20T13:56:37.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25037fe700 1 -- 172.21.15.117:0/2462773630 learned_addr learned my addr 172.21.15.117:0/2462773630 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:37.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f2502ffd700 1 --2- 172.21.15.117:0/2462773630 >> v2:172.21.15.85:3300/0 conn(0x7f250409bd80 0x7f250409cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25037fe700 1 -- 172.21.15.117:0/2462773630 >> v2:172.21.15.85:3300/0 conn(0x7f250409bd80 msgr2=0x7f250409cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25037fe700 1 --2- 172.21.15.117:0/2462773630 >> v2:172.21.15.85:3300/0 conn(0x7f250409bd80 0x7f250409cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25037fe700 1 -- 172.21.15.117:0/2462773630 >> v1:172.21.15.117:6789/0 conn(0x7f250409a110 legacy=0x7f25041437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25037fe700 1 -- 172.21.15.117:0/2462773630 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250409a5c0 con 0x7f2504143eb0 2020-11-20T13:56:37.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f2501ffb700 1 -- 172.21.15.117:0/2462773630 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466228522 0 0) 0x7f25040999d0 con 0x7f250409a110 2020-11-20T13:56:37.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f25037fe700 1 --2- 172.21.15.117:0/2462773630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f25040b4110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f24fc0032b0 tx=0x7f24fc009270).ready entity=mon.1 client_cookie=8e8b392f728433fd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f2501ffb700 1 -- 172.21.15.117:0/2462773630 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc009e60 con 0x7f2504143eb0 2020-11-20T13:56:37.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f2501ffb700 1 -- 172.21.15.117:0/2462773630 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc00bc00 con 0x7f2504143eb0 2020-11-20T13:56:37.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f2501ffb700 1 -- 172.21.15.117:0/2462773630 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc00bd70 con 0x7f2504143eb0 2020-11-20T13:56:37.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f250959b700 1 -- 172.21.15.117:0/2462773630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 msgr2=0x7f25040b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.470+0000 7f250959b700 1 --2- 172.21.15.117:0/2462773630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f25040b4110 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f24fc0032b0 tx=0x7f24fc009270).stop 2020-11-20T13:56:37.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.471+0000 7f250959b700 1 -- 172.21.15.117:0/2462773630 shutdown_connections 2020-11-20T13:56:37.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.471+0000 7f250959b700 1 --2- 172.21.15.117:0/2462773630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f25040b4110 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.471+0000 7f250959b700 1 --2- 172.21.15.117:0/2462773630 >> v2:172.21.15.85:3300/0 conn(0x7f250409bd80 0x7f250409cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.471+0000 7f250959b700 1 -- 172.21.15.117:0/2462773630 >> 172.21.15.117:0/2462773630 conn(0x7f25040b9770 msgr2=0x7f25040b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.471+0000 7f250959b700 1 -- 172.21.15.117:0/2462773630 shutdown_connections 2020-11-20T13:56:37.476 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.471+0000 7f250959b700 1 -- 172.21.15.117:0/2462773630 wait complete. 2020-11-20T13:56:37.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 Processor -- start 2020-11-20T13:56:37.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 -- start start 2020-11-20T13:56:37.479 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f250409a110 0x7f2504156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f250409bd80 0x7f25040b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f2504159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504159700 con 0x7f250409bd80 2020-11-20T13:56:37.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25041424a0 con 0x7f2504143eb0 2020-11-20T13:56:37.480 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f250959b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2504159930 con 0x7f250409a110 2020-11-20T13:56:37.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f25037fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f2504159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f25037fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f2504159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41846/0 (socket says 172.21.15.117:41846) 2020-11-20T13:56:37.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f25037fe700 1 -- 172.21.15.117:0/4032808958 learned_addr learned my addr 172.21.15.117:0/4032808958 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:37.481 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f25027fc700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f250409bd80 0x7f25040b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.472+0000 7f2502ffd700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f250409a110 0x7f2504156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f25037fe700 1 -- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f250409a110 msgr2=0x7f2504156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f25037fe700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f250409a110 0x7f2504156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.482 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f25037fe700 1 -- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f250409bd80 msgr2=0x7f25040b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f25037fe700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f250409bd80 0x7f25040b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f25037fe700 1 -- 172.21.15.117:0/4032808958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040b4530 con 0x7f2504143eb0 2020-11-20T13:56:37.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f25037fe700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f2504159dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f24fc0032b0 tx=0x7f24fc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f24f3fff700 1 -- 172.21.15.117:0/4032808958 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc0105b0 con 0x7f2504143eb0 2020-11-20T13:56:37.483 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f24f3fff700 1 -- 172.21.15.117:0/4032808958 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc010710 con 0x7f2504143eb0 2020-11-20T13:56:37.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f24f3fff700 1 -- 172.21.15.117:0/4032808958 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24fc010a70 con 0x7f2504143eb0 2020-11-20T13:56:37.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2504142040 con 0x7f2504143eb0 2020-11-20T13:56:37.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.473+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250415a480 con 0x7f2504143eb0 2020-11-20T13:56:37.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.474+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25041422f0 con 0x7f2504143eb0 2020-11-20T13:56:37.484 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.474+0000 7f24f3fff700 1 -- 172.21.15.117:0/4032808958 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f24fc026030 con 0x7f2504143eb0 2020-11-20T13:56:37.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.474+0000 7f24f3fff700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f24e0051320 0x7f24e00537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.474+0000 7f24f3fff700 1 -- 172.21.15.117:0/4032808958 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f24fc06a290 con 0x7f2504143eb0 2020-11-20T13:56:37.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.474+0000 7f2502ffd700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f24e0051320 0x7f24e00537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.475+0000 7f2502ffd700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f24e0051320 0x7f24e00537e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f24f800b000 tx=0x7f24f8006dc0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.485 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.478+0000 7f24f3fff700 1 -- 172.21.15.117:0/4032808958 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f24fc0424b0 con 0x7f2504143eb0 2020-11-20T13:56:37.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.627+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2504159af0 con 0x7f2504143eb0 2020-11-20T13:56:37.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.627+0000 7f24f3fff700 1 -- 172.21.15.117:0/4032808958 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v35) v1 ==== 110+0+10 (secure 0 0 0) 0x7f24fc0240b0 con 0x7f2504143eb0 2020-11-20T13:56:37.628 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:37.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f24e0051320 msgr2=0x7f24e00537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f24e0051320 0x7f24e00537e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f24f800b000 tx=0x7f24f8006dc0).stop 2020-11-20T13:56:37.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 msgr2=0x7f2504159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f2504159dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f24fc0032b0 tx=0x7f24fc016000).stop 2020-11-20T13:56:37.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 shutdown_connections 2020-11-20T13:56:37.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f24e0051320 0x7f24e00537e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f250409a110 0x7f2504156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2504143eb0 0x7f2504159dd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 --2- 172.21.15.117:0/4032808958 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f250409bd80 0x7f25040b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 >> 172.21.15.117:0/4032808958 conn(0x7f25040b9770 msgr2=0x7f250409f070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 shutdown_connections 2020-11-20T13:56:37.631 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.629+0000 7f250959b700 1 -- 172.21.15.117:0/4032808958 wait complete. 2020-11-20T13:56:37.639 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:37.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.748+0000 7f6c0b72d700 1 Processor -- start 2020-11-20T13:56:37.748 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.748+0000 7f6c0b72d700 1 -- start start 2020-11-20T13:56:37.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c0b72d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f6c0409c740 0x7f6c0409cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c0b72d700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0409d0e0 con 0x7f6c0409b130 2020-11-20T13:56:37.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c0b72d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c040b4860 con 0x7f6c04098d60 2020-11-20T13:56:37.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c0b72d700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c041439f0 con 0x7f6c0409c740 2020-11-20T13:56:37.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c094c9700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6c04098d60 0x7f6c040b5240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32778/0 (socket says 172.21.15.117:32778) 2020-11-20T13:56:37.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c094c9700 1 -- 172.21.15.117:0/2927744577 learned_addr learned my addr 172.21.15.117:0/2927744577 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:37.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c09cca700 1 --2- 172.21.15.117:0/2927744577 >> v2:172.21.15.85:3300/0 conn(0x7f6c0409c740 0x7f6c0409cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654234747 0 0) 0x7f6c040b4860 con 0x7f6c04098d60 2020-11-20T13:56:37.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bec003230 con 0x7f6c04098d60 2020-11-20T13:56:37.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 777952321 0 0) 0x7f6bec003230 con 0x7f6c04098d60 2020-11-20T13:56:37.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 >> v2:172.21.15.85:3300/0 conn(0x7f6c0409c740 msgr2=0x7f6c0409cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 --2- 172.21.15.117:0/2927744577 >> v2:172.21.15.85:3300/0 conn(0x7f6c0409c740 0x7f6c0409cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 >> v1:172.21.15.85:6789/0 conn(0x7f6c0409b130 legacy=0x7f6c040630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040b4b00 con 0x7f6c04098d60 2020-11-20T13:56:37.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 510105767 0 0) 0x7f6bf40016d0 con 0x7f6c04098d60 2020-11-20T13:56:37.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 >> v1:172.21.15.117:6789/0 conn(0x7f6c04098d60 legacy=0x7f6c040b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec003720 0x7f6bec003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6bec0044d0 0x7f6bec004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec004e40 0x7f6bec00b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c041439f0 con 0x7f6bec0044d0 2020-11-20T13:56:37.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec004340 con 0x7f6bec003720 2020-11-20T13:56:37.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec00bc40 con 0x7f6bec004e40 2020-11-20T13:56:37.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c094c9700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec003720 0x7f6bec003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.749+0000 7f6c08cc8700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6bec0044d0 0x7f6bec004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c09cca700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec004e40 0x7f6bec00b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c094c9700 1 -- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec004e40 msgr2=0x7f6bec00b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c094c9700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec004e40 0x7f6bec00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c094c9700 1 -- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6bec0044d0 msgr2=0x7f6bec004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:37.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c094c9700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6bec0044d0 0x7f6bec004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c09cca700 1 -- 172.21.15.117:0/2927744577 reap_dead start 2020-11-20T13:56:37.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c094c9700 1 -- 172.21.15.117:0/2927744577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec003230 con 0x7f6bec003720 2020-11-20T13:56:37.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c094c9700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec003720 0x7f6bec003b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f6bf40013e0 tx=0x7f6bf40076c0).ready entity=mon.1 client_cookie=668613513ea5b9d2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf40070d0 con 0x7f6bec003720 2020-11-20T13:56:37.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf4007230 con 0x7f6bec003720 2020-11-20T13:56:37.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6bf3fff700 1 -- 172.21.15.117:0/2927744577 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf40055e0 con 0x7f6bec003720 2020-11-20T13:56:37.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c0b72d700 1 -- 172.21.15.117:0/2927744577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec003720 msgr2=0x7f6bec003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec003720 0x7f6bec003b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f6bf40013e0 tx=0x7f6bf40076c0).stop 2020-11-20T13:56:37.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c0b72d700 1 -- 172.21.15.117:0/2927744577 shutdown_connections 2020-11-20T13:56:37.759 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/2927744577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bec003720 0x7f6bec003b40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.750+0000 7f6c0b72d700 1 -- 172.21.15.117:0/2927744577 >> 172.21.15.117:0/2927744577 conn(0x7f6c040b9770 msgr2=0x7f6c040b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.751+0000 7f6c0b72d700 1 -- 172.21.15.117:0/2927744577 shutdown_connections 2020-11-20T13:56:37.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.751+0000 7f6c0b72d700 1 -- 172.21.15.117:0/2927744577 wait complete. 2020-11-20T13:56:37.760 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.751+0000 7f6c0b72d700 1 Processor -- start 2020-11-20T13:56:37.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.751+0000 7f6c0b72d700 1 -- start start 2020-11-20T13:56:37.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c0b72d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec003720 0x7f6c04142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c0b72d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c040641c0 0x7f6c040af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c0b72d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6c04065d20 0x7f6c04136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c0b72d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040b2f80 con 0x7f6c04065d20 2020-11-20T13:56:37.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c0b72d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04066980 con 0x7f6c040641c0 2020-11-20T13:56:37.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c0b72d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04135770 con 0x7f6bec003720 2020-11-20T13:56:37.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c08cc8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c040641c0 0x7f6c040af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c08cc8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c040641c0 0x7f6c040af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41862/0 (socket says 172.21.15.117:41862) 2020-11-20T13:56:37.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c08cc8700 1 -- 172.21.15.117:0/781691974 learned_addr learned my addr 172.21.15.117:0/781691974 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:37.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c094c9700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec003720 0x7f6c04142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c09cca700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6c04065d20 0x7f6c04136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c08cc8700 1 -- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec003720 msgr2=0x7f6c04142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.752+0000 7f6c08cc8700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec003720 0x7f6c04142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6c08cc8700 1 -- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6c04065d20 msgr2=0x7f6c04136d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6c08cc8700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6c04065d20 0x7f6c04136d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6c08cc8700 1 -- 172.21.15.117:0/781691974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04135a10 con 0x7f6c040641c0 2020-11-20T13:56:37.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6c08cc8700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c040641c0 0x7f6c040af3c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f6c04067420 tx=0x7f6c00007b60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6bf27fc700 1 -- 172.21.15.117:0/781691974 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c00007240 con 0x7f6c040641c0 2020-11-20T13:56:37.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6bf27fc700 1 -- 172.21.15.117:0/781691974 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c000073a0 con 0x7f6c040641c0 2020-11-20T13:56:37.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c040b31b0 con 0x7f6c040641c0 2020-11-20T13:56:37.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6bf27fc700 1 -- 172.21.15.117:0/781691974 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c000066d0 con 0x7f6c040641c0 2020-11-20T13:56:37.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.753+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c04137240 con 0x7f6c040641c0 2020-11-20T13:56:37.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.754+0000 7f6bf27fc700 1 -- 172.21.15.117:0/781691974 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f6c00006830 con 0x7f6c040641c0 2020-11-20T13:56:37.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.754+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c040af200 con 0x7f6c040641c0 2020-11-20T13:56:37.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.754+0000 7f6bf27fc700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6be40513c0 0x7f6be4053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:37.768 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.754+0000 7f6bf27fc700 1 -- 172.21.15.117:0/781691974 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f6c00069550 con 0x7f6c040641c0 2020-11-20T13:56:37.769 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.754+0000 7f6c094c9700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6be40513c0 0x7f6be4053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:37.769 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.754+0000 7f6c094c9700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6be40513c0 0x7f6be4053880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6bf401a000 tx=0x7f6bf4015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:37.769 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.757+0000 7f6bf27fc700 1 -- 172.21.15.117:0/781691974 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f6c00041660 con 0x7f6c040641c0 2020-11-20T13:56:37.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.908+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6c04057050 con 0x7f6c040641c0 2020-11-20T13:56:37.909 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.908+0000 7f6bf27fc700 1 -- 172.21.15.117:0/781691974 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6c00044c80 con 0x7f6c040641c0 2020-11-20T13:56:37.909 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:37.910 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6be40513c0 msgr2=0x7f6be4053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6be40513c0 0x7f6be4053880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6bf401a000 tx=0x7f6bf4015000).stop 2020-11-20T13:56:37.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c040641c0 msgr2=0x7f6c040af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:37.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c040641c0 0x7f6c040af3c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f6c04067420 tx=0x7f6c00007b60).stop 2020-11-20T13:56:37.911 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 shutdown_connections 2020-11-20T13:56:37.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f6be40513c0 0x7f6be4053880 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f6bec003720 0x7f6c04142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f6c04065d20 0x7f6c04136d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.912 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 --2- 172.21.15.117:0/781691974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c040641c0 0x7f6c040af3c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:37.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 >> 172.21.15.117:0/781691974 conn(0x7f6c040b9770 msgr2=0x7f6c040b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:37.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 shutdown_connections 2020-11-20T13:56:37.913 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:37.910+0000 7f6c0b72d700 1 -- 172.21.15.117:0/781691974 wait complete. 2020-11-20T13:56:37.922 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.031+0000 7f9fb90f5700 1 Processor -- start 2020-11-20T13:56:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.031+0000 7f9fb90f5700 1 -- start start 2020-11-20T13:56:38.033 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.031+0000 7f9fb90f5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb409cdd0 0x7f9fb4147c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.031+0000 7f9fb90f5700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb4066890 con 0x7f9fb405ee90 2020-11-20T13:56:38.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.031+0000 7f9fb90f5700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb4066ac0 con 0x7f9fb4146840 2020-11-20T13:56:38.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.031+0000 7f9fb90f5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4148140 con 0x7f9fb409cdd0 2020-11-20T13:56:38.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.031+0000 7f9fb88f4700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9fb4146840 0x7f9fb409e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32800/0 (socket says 172.21.15.117:32800) 2020-11-20T13:56:38.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb88f4700 1 -- 172.21.15.117:0/1010360913 learned_addr learned my addr 172.21.15.117:0/1010360913 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:38.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb3fff700 1 --2- 172.21.15.117:0/1010360913 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb409cdd0 0x7f9fb4147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb2ffd700 1 -- 172.21.15.117:0/1010360913 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989099504 0 0) 0x7f9fb4066ac0 con 0x7f9fb4146840 2020-11-20T13:56:38.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb2ffd700 1 -- 172.21.15.117:0/1010360913 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa0003230 con 0x7f9fb4146840 2020-11-20T13:56:38.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb2ffd700 1 -- 172.21.15.117:0/1010360913 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733508643 0 0) 0x7f9fb4066890 con 0x7f9fb405ee90 2020-11-20T13:56:38.035 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb3fff700 1 -- 172.21.15.117:0/1010360913 >> v1:172.21.15.117:6789/0 conn(0x7f9fb4146840 legacy=0x7f9fb409e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb3fff700 1 -- 172.21.15.117:0/1010360913 >> v1:172.21.15.85:6789/0 conn(0x7f9fb405ee90 legacy=0x7f9fb4067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb3fff700 1 -- 172.21.15.117:0/1010360913 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb4066d60 con 0x7f9fb409cdd0 2020-11-20T13:56:38.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb3fff700 1 --2- 172.21.15.117:0/1010360913 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb409cdd0 0x7f9fb4147c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9fa4002f30 tx=0x7f9fa4006dc0).ready entity=mon.2 client_cookie=7696d4e2f68b7481 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb2ffd700 1 -- 172.21.15.117:0/1010360913 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa40097b0 con 0x7f9fb409cdd0 2020-11-20T13:56:38.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb2ffd700 1 -- 172.21.15.117:0/1010360913 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa4009910 con 0x7f9fb409cdd0 2020-11-20T13:56:38.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb2ffd700 1 -- 172.21.15.117:0/1010360913 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa4009c70 con 0x7f9fb409cdd0 2020-11-20T13:56:38.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb90f5700 1 -- 172.21.15.117:0/1010360913 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb409cdd0 msgr2=0x7f9fb4147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.032+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/1010360913 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb409cdd0 0x7f9fb4147c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9fa4002f30 tx=0x7f9fa4006dc0).stop 2020-11-20T13:56:38.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.033+0000 7f9fb90f5700 1 -- 172.21.15.117:0/1010360913 shutdown_connections 2020-11-20T13:56:38.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.033+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/1010360913 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb409cdd0 0x7f9fb4147c50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.033+0000 7f9fb90f5700 1 -- 172.21.15.117:0/1010360913 >> 172.21.15.117:0/1010360913 conn(0x7f9fb40bd890 msgr2=0x7f9fb40bdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.033+0000 7f9fb90f5700 1 -- 172.21.15.117:0/1010360913 shutdown_connections 2020-11-20T13:56:38.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.033+0000 7f9fb90f5700 1 -- 172.21.15.117:0/1010360913 wait complete. 2020-11-20T13:56:38.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.034+0000 7f9fb90f5700 1 Processor -- start 2020-11-20T13:56:38.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.034+0000 7f9fb90f5700 1 -- start start 2020-11-20T13:56:38.042 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb405ee90 0x7f9fb40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9fb409cdd0 0x7f9fb4159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb4146840 0x7f9fb40b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40a19e0 con 0x7f9fb409cdd0 2020-11-20T13:56:38.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40a1c80 con 0x7f9fb405ee90 2020-11-20T13:56:38.043 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40b2b80 con 0x7f9fb4146840 2020-11-20T13:56:38.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb405ee90 0x7f9fb40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.044 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb405ee90 0x7f9fb40b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41874/0 (socket says 172.21.15.117:41874) 2020-11-20T13:56:38.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 -- 172.21.15.117:0/3055023821 learned_addr learned my addr 172.21.15.117:0/3055023821 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:38.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb88f4700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb4146840 0x7f9fb40b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.045 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb37fe700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9fb409cdd0 0x7f9fb4159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.046 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 -- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb4146840 msgr2=0x7f9fb40b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.046 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb4146840 0x7f9fb40b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.046 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 -- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9fb409cdd0 msgr2=0x7f9fb4159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.046 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9fb409cdd0 0x7f9fb4159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.046 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 -- 172.21.15.117:0/3055023821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40b76a0 con 0x7f9fb405ee90 2020-11-20T13:56:38.047 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb3fff700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb405ee90 0x7f9fb40b71b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f9fa4002f30 tx=0x7f9fa4015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.047 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb17fa700 1 -- 172.21.15.117:0/3055023821 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa4009e20 con 0x7f9fb405ee90 2020-11-20T13:56:38.047 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb17fa700 1 -- 172.21.15.117:0/3055023821 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa401d450 con 0x7f9fb405ee90 2020-11-20T13:56:38.047 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb4159f10 con 0x7f9fb405ee90 2020-11-20T13:56:38.047 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb17fa700 1 -- 172.21.15.117:0/3055023821 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa401d7b0 con 0x7f9fb405ee90 2020-11-20T13:56:38.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.035+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb415a450 con 0x7f9fb405ee90 2020-11-20T13:56:38.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.036+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb40b7830 con 0x7f9fb405ee90 2020-11-20T13:56:38.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.036+0000 7f9fb17fa700 1 -- 172.21.15.117:0/3055023821 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9fa400f070 con 0x7f9fb405ee90 2020-11-20T13:56:38.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.037+0000 7f9fb17fa700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f94051290 0x7f9f94053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.037+0000 7f9fb17fa700 1 -- 172.21.15.117:0/3055023821 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9fa406ad10 con 0x7f9fb405ee90 2020-11-20T13:56:38.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.037+0000 7f9fb37fe700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f94051290 0x7f9f94053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.037+0000 7f9fb37fe700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f94051290 0x7f9f94053750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f9fa800b000 tx=0x7f9fa8009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.040+0000 7f9fb17fa700 1 -- 172.21.15.117:0/3055023821 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9fa40468c0 con 0x7f9fb405ee90 2020-11-20T13:56:38.193 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.191+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fb4057050 con 0x7f9fb405ee90 2020-11-20T13:56:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.191+0000 7f9fb17fa700 1 -- 172.21.15.117:0/3055023821 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9fa40464b0 con 0x7f9fb405ee90 2020-11-20T13:56:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f94051290 msgr2=0x7f9f94053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f94051290 0x7f9f94053750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f9fa800b000 tx=0x7f9fa8009040).stop 2020-11-20T13:56:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb405ee90 msgr2=0x7f9fb40b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb405ee90 0x7f9fb40b71b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f9fa4002f30 tx=0x7f9fa4015000).stop 2020-11-20T13:56:38.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 shutdown_connections 2020-11-20T13:56:38.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9f94051290 0x7f9f94053750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9fb4146840 0x7f9fb40b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.195 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb405ee90 0x7f9fb40b71b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 --2- 172.21.15.117:0/3055023821 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9fb409cdd0 0x7f9fb4159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 >> 172.21.15.117:0/3055023821 conn(0x7f9fb40bd890 msgr2=0x7f9fb4148290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 shutdown_connections 2020-11-20T13:56:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.192+0000 7f9fb90f5700 1 -- 172.21.15.117:0/3055023821 wait complete. 2020-11-20T13:56:38.196 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:38.203 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:38.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.311+0000 7f1f7bf4c700 1 Processor -- start 2020-11-20T13:56:38.312 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.312+0000 7f1f7bf4c700 1 -- start start 2020-11-20T13:56:38.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.312+0000 7f1f7bf4c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f74098d60 0x7f1f7409b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.320 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.312+0000 7f1f7bf4c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f740b5240 0x7f1f740630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.312+0000 7f1f7bf4c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f1f740635b0 0x7f1f74062830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.312+0000 7f1f7bf4c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f741439f0 con 0x7f1f740635b0 2020-11-20T13:56:38.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.312+0000 7f1f7bf4c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740b4860 con 0x7f1f740b5240 2020-11-20T13:56:38.321 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.312+0000 7f1f7bf4c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b4b00 con 0x7f1f74098d60 2020-11-20T13:56:38.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f740b5240 0x7f1f740630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f740b5240 0x7f1f740630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41882/0 (socket says 172.21.15.117:41882) 2020-11-20T13:56:38.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 -- 172.21.15.117:0/748634327 learned_addr learned my addr 172.21.15.117:0/748634327 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:38.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f79ce8700 1 --2- 172.21.15.117:0/748634327 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f74098d60 0x7f1f7409b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f7a4e9700 1 --2- 172.21.15.117:0/748634327 >> v2:172.21.15.85:3300/0 conn(0x7f1f740635b0 0x7f1f74062830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 -- 172.21.15.117:0/748634327 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f74098d60 msgr2=0x7f1f7409b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:38.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 --2- 172.21.15.117:0/748634327 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f74098d60 0x7f1f7409b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 -- 172.21.15.117:0/748634327 >> v2:172.21.15.85:3300/0 conn(0x7f1f740635b0 msgr2=0x7f1f74062830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:38.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 --2- 172.21.15.117:0/748634327 >> v2:172.21.15.85:3300/0 conn(0x7f1f740635b0 0x7f1f74062830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 -- 172.21.15.117:0/748634327 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740b4da0 con 0x7f1f740b5240 2020-11-20T13:56:38.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f794e7700 1 --2- 172.21.15.117:0/748634327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f740b5240 0x7f1f740630c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f1f74143b30 tx=0x7f1f70006e80).ready entity=mon.1 client_cookie=aced8c5d8b812a19 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f78ce6700 1 -- 172.21.15.117:0/748634327 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f700097b0 con 0x7f1f740b5240 2020-11-20T13:56:38.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f78ce6700 1 -- 172.21.15.117:0/748634327 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70009910 con 0x7f1f740b5240 2020-11-20T13:56:38.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f78ce6700 1 -- 172.21.15.117:0/748634327 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f70009c70 con 0x7f1f740b5240 2020-11-20T13:56:38.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/748634327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f740b5240 msgr2=0x7f1f740630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.313+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/748634327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f740b5240 0x7f1f740630c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f1f74143b30 tx=0x7f1f70006e80).stop 2020-11-20T13:56:38.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.314+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/748634327 shutdown_connections 2020-11-20T13:56:38.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.314+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/748634327 >> v2:172.21.15.85:3300/0 conn(0x7f1f740635b0 0x7f1f74062830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.314+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/748634327 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f74098d60 0x7f1f7409b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.314+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/748634327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f740b5240 0x7f1f740630c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.314+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/748634327 >> 172.21.15.117:0/748634327 conn(0x7f1f740b9770 msgr2=0x7f1f740b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.314+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/748634327 shutdown_connections 2020-11-20T13:56:38.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.314+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/748634327 wait complete. 2020-11-20T13:56:38.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 Processor -- start 2020-11-20T13:56:38.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 -- start start 2020-11-20T13:56:38.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f740635b0 0x7f1f74067640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f74098d60 0x7f1f740afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f740b5240 0x7f1f7413bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f74064cd0 con 0x7f1f740635b0 2020-11-20T13:56:38.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b2710 con 0x7f1f74098d60 2020-11-20T13:56:38.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f7bf4c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b29b0 con 0x7f1f740b5240 2020-11-20T13:56:38.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f794e7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f74098d60 0x7f1f740afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f794e7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f74098d60 0x7f1f740afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41888/0 (socket says 172.21.15.117:41888) 2020-11-20T13:56:38.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.315+0000 7f1f794e7700 1 -- 172.21.15.117:0/747079967 learned_addr learned my addr 172.21.15.117:0/747079967 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:38.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f79ce8700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f740635b0 0x7f1f74067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f7a4e9700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f740b5240 0x7f1f7413bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f794e7700 1 -- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f740b5240 msgr2=0x7f1f7413bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f794e7700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f740b5240 0x7f1f7413bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f794e7700 1 -- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f740635b0 msgr2=0x7f1f74067640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f794e7700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f740635b0 0x7f1f74067640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f794e7700 1 -- 172.21.15.117:0/747079967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740b23c0 con 0x7f1f74098d60 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f794e7700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f74098d60 0x7f1f740afc50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f1f74143b30 tx=0x7f1f70015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f62ffd700 1 -- 172.21.15.117:0/747079967 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f70011ce0 con 0x7f1f74098d60 2020-11-20T13:56:38.331 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f62ffd700 1 -- 172.21.15.117:0/747079967 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70011e40 con 0x7f1f74098d60 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740b03e0 con 0x7f1f74098d60 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f62ffd700 1 -- 172.21.15.117:0/747079967 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f7001e650 con 0x7f1f74098d60 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.316+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7413c4e0 con 0x7f1f74098d60 2020-11-20T13:56:38.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.317+0000 7f1f62ffd700 1 -- 172.21.15.117:0/747079967 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1f70025030 con 0x7f1f74098d60 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.317+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f74057050 con 0x7f1f74098d60 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.317+0000 7f1f62ffd700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f54051210 0x7f1f540536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.317+0000 7f1f62ffd700 1 -- 172.21.15.117:0/747079967 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f1f7006a5c0 con 0x7f1f74098d60 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.317+0000 7f1f79ce8700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f54051210 0x7f1f540536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.318+0000 7f1f79ce8700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f54051210 0x7f1f540536d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f1f6400c000 tx=0x7f1f64008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.321+0000 7f1f62ffd700 1 -- 172.21.15.117:0/747079967 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1f700427e0 con 0x7f1f74098d60 2020-11-20T13:56:38.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.469+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f7414ef80 con 0x7f1f74098d60 2020-11-20T13:56:38.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.469+0000 7f1f62ffd700 1 -- 172.21.15.117:0/747079967 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1f70023020 con 0x7f1f74098d60 2020-11-20T13:56:38.470 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:38.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f54051210 msgr2=0x7f1f540536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f54051210 0x7f1f540536d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f1f6400c000 tx=0x7f1f64008040).stop 2020-11-20T13:56:38.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f74098d60 msgr2=0x7f1f740afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f74098d60 0x7f1f740afc50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f1f74143b30 tx=0x7f1f70015000).stop 2020-11-20T13:56:38.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 shutdown_connections 2020-11-20T13:56:38.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1f54051210 0x7f1f540536d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1f740b5240 0x7f1f7413bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.472 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1f740635b0 0x7f1f74067640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.473 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 --2- 172.21.15.117:0/747079967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f74098d60 0x7f1f740afc50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 >> 172.21.15.117:0/747079967 conn(0x7f1f740b9770 msgr2=0x7f1f7409b680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.474 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 shutdown_connections 2020-11-20T13:56:38.475 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.471+0000 7f1f7bf4c700 1 -- 172.21.15.117:0/747079967 wait complete. 2020-11-20T13:56:38.481 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:38.481 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:38.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.588+0000 7f18f3d0d700 1 Processor -- start 2020-11-20T13:56:38.588 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.588+0000 7f18f3d0d700 1 -- start start 2020-11-20T13:56:38.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18ec0b0080 0x7f18ec146420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.591 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f18ec09bfe0 0x7f18ec148a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec148f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec098be0 con 0x7f18ec09bfe0 2020-11-20T13:56:38.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec098e80 con 0x7f18ec0b0080 2020-11-20T13:56:38.592 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec099120 con 0x7f18ec09c7a0 2020-11-20T13:56:38.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18ec0b0080 0x7f18ec146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18ec0b0080 0x7f18ec146420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41894/0 (socket says 172.21.15.117:41894) 2020-11-20T13:56:38.593 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3774682603 learned_addr learned my addr 172.21.15.117:0/3774682603 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:38.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f22aa700 1 --2- 172.21.15.117:0/3774682603 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f12a8700 1 --2- 172.21.15.117:0/3774682603 >> v2:172.21.15.85:3300/0 conn(0x7f18ec09bfe0 0x7f18ec148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3774682603 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 msgr2=0x7f18ec148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:38.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 --2- 172.21.15.117:0/3774682603 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.594 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3774682603 >> v2:172.21.15.85:3300/0 conn(0x7f18ec09bfe0 msgr2=0x7f18ec148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:38.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 --2- 172.21.15.117:0/3774682603 >> v2:172.21.15.85:3300/0 conn(0x7f18ec09bfe0 0x7f18ec148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3774682603 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0b0460 con 0x7f18ec0b0080 2020-11-20T13:56:38.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f1aa9700 1 --2- 172.21.15.117:0/3774682603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec0b0080 0x7f18ec146420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f18e40032b0 tx=0x7f18e4009270).ready entity=mon.1 client_cookie=b6f3b11f1616d8d0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f0aa7700 1 -- 172.21.15.117:0/3774682603 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e400bc00 con 0x7f18ec0b0080 2020-11-20T13:56:38.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f0aa7700 1 -- 172.21.15.117:0/3774682603 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e400bd60 con 0x7f18ec0b0080 2020-11-20T13:56:38.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f0aa7700 1 -- 172.21.15.117:0/3774682603 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e4011400 con 0x7f18ec0b0080 2020-11-20T13:56:38.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3774682603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec0b0080 msgr2=0x7f18ec146420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.589+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3774682603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec0b0080 0x7f18ec146420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f18e40032b0 tx=0x7f18e4009270).stop 2020-11-20T13:56:38.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.590+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3774682603 shutdown_connections 2020-11-20T13:56:38.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.590+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3774682603 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.590+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3774682603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec0b0080 0x7f18ec146420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.590+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3774682603 >> v2:172.21.15.85:3300/0 conn(0x7f18ec09bfe0 0x7f18ec148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.590+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3774682603 >> 172.21.15.117:0/3774682603 conn(0x7f18ec05b9f0 msgr2=0x7f18ec05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.590+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3774682603 shutdown_connections 2020-11-20T13:56:38.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.590+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3774682603 wait complete. 2020-11-20T13:56:38.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.591+0000 7f18f3d0d700 1 Processor -- start 2020-11-20T13:56:38.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.591+0000 7f18f3d0d700 1 -- start start 2020-11-20T13:56:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec09bfe0 0x7f18ec066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec099400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18ec0b0080 0x7f18ec0998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0b8a00 con 0x7f18ec0b0080 2020-11-20T13:56:38.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0b8ca0 con 0x7f18ec09bfe0 2020-11-20T13:56:38.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0b8f40 con 0x7f18ec09c7a0 2020-11-20T13:56:38.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec09bfe0 0x7f18ec066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec09bfe0 0x7f18ec066f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41900/0 (socket says 172.21.15.117:41900) 2020-11-20T13:56:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3097855597 learned_addr learned my addr 172.21.15.117:0/3097855597 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f12a8700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3097855597 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 msgr2=0x7f18ec099400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec099400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3097855597 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18ec0b0080 msgr2=0x7f18ec0998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18ec0b0080 0x7f18ec0998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:38.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 -- 172.21.15.117:0/3097855597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0b99d0 con 0x7f18ec09bfe0 2020-11-20T13:56:38.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f1aa9700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec09bfe0 0x7f18ec066f20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f18e40032b0 tx=0x7f18e4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18deffd700 1 -- 172.21.15.117:0/3097855597 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e401f5e0 con 0x7f18ec09bfe0 2020-11-20T13:56:38.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18deffd700 1 -- 172.21.15.117:0/3097855597 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e401f740 con 0x7f18ec09bfe0 2020-11-20T13:56:38.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18deffd700 1 -- 172.21.15.117:0/3097855597 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18e401faa0 con 0x7f18ec09bfe0 2020-11-20T13:56:38.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec0b9b60 con 0x7f18ec09bfe0 2020-11-20T13:56:38.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.592+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec0b87a0 con 0x7f18ec09bfe0 2020-11-20T13:56:38.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.594+0000 7f18deffd700 1 -- 172.21.15.117:0/3097855597 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f18e400f070 con 0x7f18ec09bfe0 2020-11-20T13:56:38.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.594+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec057050 con 0x7f18ec09bfe0 2020-11-20T13:56:38.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.594+0000 7f18deffd700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f18cc051210 0x7f18cc0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:38.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.594+0000 7f18deffd700 1 -- 172.21.15.117:0/3097855597 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f18e4069ed0 con 0x7f18ec09bfe0 2020-11-20T13:56:38.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.594+0000 7f18f12a8700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f18cc051210 0x7f18cc0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:38.607 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.594+0000 7f18f12a8700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f18cc051210 0x7f18cc0536d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f18e0008000 tx=0x7f18e0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:38.608 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.597+0000 7f18deffd700 1 -- 172.21.15.117:0/3097855597 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f18e4042170 con 0x7f18ec09bfe0 2020-11-20T13:56:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.797+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18ec0b9100 con 0x7f18ec09bfe0 2020-11-20T13:56:38.798 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.798+0000 7f18deffd700 1 -- 172.21.15.117:0/3097855597 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f18e4024090 con 0x7f18ec09bfe0 2020-11-20T13:56:38.799 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:38.799 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/332543540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":332543540},{"type":"v1","addr":"172.21.15.85:6833","nonce":332543540}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:34.417958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4996":{"gid":4996,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/2328953218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2328953218},{"type":"v1","addr":"172.21.15.117:6835","nonce":2328953218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f18cc051210 msgr2=0x7f18cc0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.800 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f18cc051210 0x7f18cc0536d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f18e0008000 tx=0x7f18e0008040).stop 2020-11-20T13:56:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec09bfe0 msgr2=0x7f18ec066f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec09bfe0 0x7f18ec066f20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f18e40032b0 tx=0x7f18e4016000).stop 2020-11-20T13:56:38.801 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 shutdown_connections 2020-11-20T13:56:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f18cc051210 0x7f18cc0536d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18ec0b0080 0x7f18ec0998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18ec09bfe0 0x7f18ec066f20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.802 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 --2- 172.21.15.117:0/3097855597 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f18ec09c7a0 0x7f18ec099400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:38.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 >> 172.21.15.117:0/3097855597 conn(0x7f18ec05b9f0 msgr2=0x7f18ec09e310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:38.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 shutdown_connections 2020-11-20T13:56:38.803 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:38.799+0000 7f18f3d0d700 1 -- 172.21.15.117:0/3097855597 wait complete. 2020-11-20T13:56:38.803 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 10 2020-11-20T13:56:39.012 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:33.416903+0000', 'modified': '2020-11-20T13:56:34.417958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4996': {'gid': 4996, 'name': 'a', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6835/2328953218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2328953218}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2328953218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:39.013 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:39.013 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:39.141 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":4996,"want_state":"up:active","state":"up:active","rank_uptime":5.714163664,"mdsmap_epoch":10,"osdmap_epoch":35,"osdmap_epoch_barrier":34,"uptime":12.514457148} 2020-11-20T13:56:39.141 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 4996, "mdsmap_epoch": 10, "osdmap_epoch": 35, "osdmap_epoch_barrier": 34, "rank_uptime": 5.714163664, "state": "up:active", "uptime": 12.514457148, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:39.142 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:39.142 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-20T13:56:39.143 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi117.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-20T13:56:39.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:39.144 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:39.159 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:39.226 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:39.226 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:39.292 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:39.293 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:39.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:39.294 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:39.348 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-20T13:56:39.349 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:39.391 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:39.419 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:39.420 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:39.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:2020-11-20T13:56:39.467+0000 7fe57145c1c0 -1 init, newargv = 0x55d22b8da2e0 newargc=9 2020-11-20T13:56:39.468 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[30505]: starting ceph client 2020-11-20T13:56:39.471 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[30505]: starting fuse 2020-11-20T13:56:39.487 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:39.488 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:39.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:39.489 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:39.543 INFO:teuthology.orchestra.run.smithi117.stdout:47 2020-11-20T13:56:39.544 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-20T13:56:39.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:39.545 INFO:teuthology.orchestra.run.smithi117:> sudo python3 -c ' 2020-11-20T13:56:39.545 INFO:teuthology.orchestra.run.smithi117:> import glob 2020-11-20T13:56:39.545 INFO:teuthology.orchestra.run.smithi117:> import re 2020-11-20T13:56:39.546 INFO:teuthology.orchestra.run.smithi117:> import os 2020-11-20T13:56:39.546 INFO:teuthology.orchestra.run.smithi117:> import subprocess 2020-11-20T13:56:39.547 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:39.547 INFO:teuthology.orchestra.run.smithi117:> def find_socket(client_name): 2020-11-20T13:56:39.547 INFO:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-20T13:56:39.548 INFO:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2020-11-20T13:56:39.548 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:39.548 INFO:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2020-11-20T13:56:39.549 INFO:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2020-11-20T13:56:39.551 INFO:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2020-11-20T13:56:39.551 INFO:teuthology.orchestra.run.smithi117:> return files[0] 2020-11-20T13:56:39.552 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:39.553 INFO:teuthology.orchestra.run.smithi117:> for f in files: 2020-11-20T13:56:39.553 INFO:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:39.554 INFO:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:39.554 INFO:teuthology.orchestra.run.smithi117:> return f 2020-11-20T13:56:39.555 INFO:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:39.555 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:39.555 INFO:teuthology.orchestra.run.smithi117:> print(find_socket("client.0")) 2020-11-20T13:56:39.556 INFO:teuthology.orchestra.run.smithi117:> ' 2020-11-20T13:56:39.635 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.0.30505.asok 2020-11-20T13:56:39.639 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.30505.asok 2020-11-20T13:56:39.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:39.640 INFO:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.30505.asok status 2020-11-20T13:56:39.745 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2020-11-20T13:56:39.745 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2020-11-20T13:56:39.745 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:39.746 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:39.746 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "0", 2020-11-20T13:56:39.746 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2020-11-20T13:56:39.746 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-20T13:56:39.746 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "30505", 2020-11-20T13:56:39.747 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2020-11-20T13:56:39.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:39.747 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2020-11-20T13:56:39.747 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:39.747 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 5221, 2020-11-20T13:56:39.748 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2020-11-20T13:56:39.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2020-11-20T13:56:39.748 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2020-11-20T13:56:39.749 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 5221 2020-11-20T13:56:39.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:39.749 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:39.749 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:39.749 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:39.750 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2666274739 2020-11-20T13:56:39.750 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:39.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:39.751 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:39.751 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:39.751 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:39.751 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2666274739 2020-11-20T13:56:39.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:39.752 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.5221 172.21.15.117:0/2666274739", 2020-11-20T13:56:39.752 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "172.21.15.117:0/2666274739", 2020-11-20T13:56:39.752 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2020-11-20T13:56:39.755 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 10, 2020-11-20T13:56:39.756 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 35, 2020-11-20T13:56:39.756 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:39.756 INFO:teuthology.orchestra.run.smithi117.stdout: "blacklisted": false 2020-11-20T13:56:39.756 INFO:teuthology.orchestra.run.smithi117.stdout:} 2020-11-20T13:56:39.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:39.758 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:39.758 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:39.773 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:39.774 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:39.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:39.774 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:39.841 INFO:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir testdir1 2020-11-20T13:56:39.915 INFO:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.0 && sudo touch testdir1/testfile 2020-11-20T13:56:39.985 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T13:56:39.986 INFO:teuthology.orchestra.run.smithi117:> import os 2020-11-20T13:56:39.986 INFO:teuthology.orchestra.run.smithi117:> import stat 2020-11-20T13:56:39.987 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:39.987 INFO:teuthology.orchestra.run.smithi117:> print(os.stat("/home/ubuntu/cephtest/mnt.0/testdir1").st_ino) 2020-11-20T13:56:39.987 INFO:teuthology.orchestra.run.smithi117:> ' 2020-11-20T13:56:40.101 INFO:teuthology.orchestra.run.smithi117.stdout:1099511627776 2020-11-20T13:56:40.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:40.291 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:40.292 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:40.346 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:40.346 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:40.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:40.347 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:40.347 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:40.401 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:40.402 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:40.402 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi117.front.sepia.ceph.com... 2020-11-20T13:56:40.403 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T13:56:40.403 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:40.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[30505]: fuse finished with error 0 and tester_r 0 2020-11-20T13:56:40.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:40.482 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:40.482 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:40.535 INFO:teuthology.orchestra.run.smithi117.stdout:ext2/ext3 2020-11-20T13:56:40.535 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2020-11-20T13:56:40.536 INFO:teuthology.orchestra.run:waiting for 900 2020-11-20T13:56:46.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T13:56:46.638 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:46.656 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:46.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.809+0000 7f9525ec1700 1 Processor -- start 2020-11-20T13:56:46.811 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.809+0000 7f9525ec1700 1 -- start start 2020-11-20T13:56:46.813 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9525ec1700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95201473d0 0x7f95200b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:46.813 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9525ec1700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f9520067b40 0x7f95200b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:46.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9525ec1700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95200b4a00 con 0x7f95200be620 2020-11-20T13:56:46.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9525ec1700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952014af20 con 0x7f9520067b40 2020-11-20T13:56:46.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9525ec1700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952014b1c0 con 0x7f95201473d0 2020-11-20T13:56:46.814 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9517fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95201473d0 0x7f95200b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:46.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9517fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95201473d0 0x7f95200b8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41922/0 (socket says 172.21.15.117:41922) 2020-11-20T13:56:46.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9517fff700 1 -- 172.21.15.117:0/528850273 learned_addr learned my addr 172.21.15.117:0/528850273 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:46.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f951ffff700 1 --2- 172.21.15.117:0/528850273 >> v2:172.21.15.85:3300/0 conn(0x7f9520067b40 0x7f95200b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:46.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9517fff700 1 -- 172.21.15.117:0/528850273 >> v2:172.21.15.85:3300/0 conn(0x7f9520067b40 msgr2=0x7f95200b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:46.815 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9517fff700 1 --2- 172.21.15.117:0/528850273 >> v2:172.21.15.85:3300/0 conn(0x7f9520067b40 0x7f95200b8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.810+0000 7f9517fff700 1 -- 172.21.15.117:0/528850273 >> v1:172.21.15.85:6789/0 conn(0x7f95200be620 legacy=0x7f9520146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:46.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9517fff700 1 -- 172.21.15.117:0/528850273 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952014b460 con 0x7f95201473d0 2020-11-20T13:56:46.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9517fff700 1 --2- 172.21.15.117:0/528850273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95201473d0 0x7f95200b8530 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f95200b8330 tx=0x7f9508006e80).ready entity=mon.1 client_cookie=c5aba476f8c9caa6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:46.816 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f951effd700 1 -- 172.21.15.117:0/528850273 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95080097b0 con 0x7f95201473d0 2020-11-20T13:56:46.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f951effd700 1 -- 172.21.15.117:0/528850273 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9508009910 con 0x7f95201473d0 2020-11-20T13:56:46.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f951effd700 1 -- 172.21.15.117:0/528850273 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9508009a80 con 0x7f95201473d0 2020-11-20T13:56:46.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 -- 172.21.15.117:0/528850273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95201473d0 msgr2=0x7f95200b8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:46.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 --2- 172.21.15.117:0/528850273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95201473d0 0x7f95200b8530 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f95200b8330 tx=0x7f9508006e80).stop 2020-11-20T13:56:46.817 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 -- 172.21.15.117:0/528850273 shutdown_connections 2020-11-20T13:56:46.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 --2- 172.21.15.117:0/528850273 >> v2:172.21.15.85:3300/0 conn(0x7f9520067b40 0x7f95200b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 --2- 172.21.15.117:0/528850273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95201473d0 0x7f95200b8530 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 -- 172.21.15.117:0/528850273 >> 172.21.15.117:0/528850273 conn(0x7f952005c4c0 msgr2=0x7f95200bdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:46.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 -- 172.21.15.117:0/528850273 shutdown_connections 2020-11-20T13:56:46.818 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.811+0000 7f9525ec1700 1 -- 172.21.15.117:0/528850273 wait complete. 2020-11-20T13:56:46.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.812+0000 7f9525ec1700 1 Processor -- start 2020-11-20T13:56:46.819 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.812+0000 7f9525ec1700 1 -- start start 2020-11-20T13:56:46.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9525ec1700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9520067b40 0x7f9520158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:46.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9525ec1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95200be620 0x7f9520149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:46.821 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9525ec1700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f95201473d0 0x7f9520138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:46.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9525ec1700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95200a3270 con 0x7f9520067b40 2020-11-20T13:56:46.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9525ec1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952015a1a0 con 0x7f95200be620 2020-11-20T13:56:46.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9525ec1700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952015a440 con 0x7f95201473d0 2020-11-20T13:56:46.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95200be620 0x7f9520149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:46.822 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95200be620 0x7f9520149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41930/0 (socket says 172.21.15.117:41930) 2020-11-20T13:56:46.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 -- 172.21.15.117:0/836368333 learned_addr learned my addr 172.21.15.117:0/836368333 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:46.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f951f7fe700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9520067b40 0x7f9520158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:46.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 -- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f95201473d0 msgr2=0x7f9520138dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:46.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f951ffff700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f95201473d0 0x7f9520138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:46.823 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f95201473d0 0x7f9520138dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 -- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9520067b40 msgr2=0x7f9520158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:46.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9520067b40 0x7f9520158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:46.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.813+0000 7f9517fff700 1 -- 172.21.15.117:0/836368333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952015a6e0 con 0x7f95200be620 2020-11-20T13:56:46.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.814+0000 7f9517fff700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95200be620 0x7f9520149ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f95200b8330 tx=0x7f9508016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:46.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.814+0000 7f951d7fa700 1 -- 172.21.15.117:0/836368333 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95080203f0 con 0x7f95200be620 2020-11-20T13:56:46.824 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.814+0000 7f951d7fa700 1 -- 172.21.15.117:0/836368333 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9508020550 con 0x7f95200be620 2020-11-20T13:56:46.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.814+0000 7f951d7fa700 1 -- 172.21.15.117:0/836368333 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95080208b0 con 0x7f95200be620 2020-11-20T13:56:46.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.814+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9520159e20 con 0x7f95200be620 2020-11-20T13:56:46.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.814+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95200a2430 con 0x7f95200be620 2020-11-20T13:56:46.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.815+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952015a870 con 0x7f95200be620 2020-11-20T13:56:46.825 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.815+0000 7f951d7fa700 1 -- 172.21.15.117:0/836368333 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f950801e070 con 0x7f95200be620 2020-11-20T13:56:46.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.815+0000 7f951d7fa700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9504051210 0x7f95040536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:46.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.815+0000 7f951d7fa700 1 -- 172.21.15.117:0/836368333 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f9508069e00 con 0x7f95200be620 2020-11-20T13:56:46.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.815+0000 7f951f7fe700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9504051210 0x7f95040536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:46.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.815+0000 7f951f7fe700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9504051210 0x7f95040536d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9510008000 tx=0x7f9510009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:46.826 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:46.819+0000 7f951d7fa700 1 -- 172.21.15.117:0/836368333 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9508041f80 con 0x7f95200be620 2020-11-20T13:56:47.018 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.017+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9520057050 con 0x7f95200be620 2020-11-20T13:56:47.019 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.017+0000 7f951d7fa700 1 -- 172.21.15.117:0/836368333 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2136 (secure 0 0 0) 0x7f950801b090 con 0x7f95200be620 2020-11-20T13:56:47.019 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:47.019 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/332543540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":332543540},{"type":"v1","addr":"172.21.15.85:6833","nonce":332543540}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:34.417958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4996":{"gid":4996,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6835/2328953218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2328953218},{"type":"v1","addr":"172.21.15.117:6835","nonce":2328953218}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:47.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9504051210 msgr2=0x7f95040536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9504051210 0x7f95040536d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9510008000 tx=0x7f9510009040).stop 2020-11-20T13:56:47.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95200be620 msgr2=0x7f9520149ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.021 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95200be620 0x7f9520149ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f95200b8330 tx=0x7f9508016000).stop 2020-11-20T13:56:47.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 shutdown_connections 2020-11-20T13:56:47.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9504051210 0x7f95040536d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f95201473d0 0x7f9520138dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9520067b40 0x7f9520158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.022 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 --2- 172.21.15.117:0/836368333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95200be620 0x7f9520149ef0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 >> 172.21.15.117:0/836368333 conn(0x7f952005c4c0 msgr2=0x7f9520145e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 shutdown_connections 2020-11-20T13:56:47.023 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.019+0000 7f9525ec1700 1 -- 172.21.15.117:0/836368333 wait complete. 2020-11-20T13:56:47.023 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 10 2020-11-20T13:56:47.030 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2020-11-20T13:56:47.162 INFO:teuthology.orchestra.run.smithi117.stdout:{"message":"","return_code":0} 2020-11-20T13:56:47.163 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2020-11-20T13:56:47.164 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T13:56:47.164 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T13:56:47.165 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T13:56:47.165 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T13:56:47.172 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T13:56:47.172 DEBUG:teuthology.parallel:result is None 2020-11-20T13:56:47.174 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T13:56:47.174 DEBUG:teuthology.parallel:result is None 2020-11-20T13:56:47.175 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2020-11-20T13:56:47.175 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail a 2020-11-20T13:56:47.176 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2020-11-20T13:56:47.176 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mds fail b 2020-11-20T13:56:47.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7f74f321a700 1 Processor -- start 2020-11-20T13:56:47.322 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7f74f321a700 1 -- start start 2020-11-20T13:56:47.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7fb46ad5c700 1 Processor -- start 2020-11-20T13:56:47.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7fb46ad5c700 1 -- start start 2020-11-20T13:56:47.332 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7f74f321a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec147190 0x7f74ec148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7f74f321a700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f74ec09aea0 0x7f74ec061f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7f74f321a700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec147880 con 0x7f74ec0b8b50 2020-11-20T13:56:47.333 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7f74f321a700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec147b20 con 0x7f74ec09aea0 2020-11-20T13:56:47.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7f74f321a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec147dc0 con 0x7f74ec147190 2020-11-20T13:56:47.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74f17b7700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f74ec0b8b50 0x7f74ec14acb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32868/0 (socket says 172.21.15.117:32868) 2020-11-20T13:56:47.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74f17b7700 1 -- 172.21.15.117:0/109380329 learned_addr learned my addr 172.21.15.117:0/109380329 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:47.334 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74f0fb6700 1 --2- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec147190 0x7f74ec148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74ebfff700 1 --2- 172.21.15.117:0/109380329 >> v2:172.21.15.85:3300/0 conn(0x7f74ec09aea0 0x7f74ec061f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74eb7fe700 1 -- 172.21.15.117:0/109380329 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235128657 0 0) 0x7f74ec147880 con 0x7f74ec0b8b50 2020-11-20T13:56:47.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74eb7fe700 1 -- 172.21.15.117:0/109380329 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74d0003230 con 0x7f74ec0b8b50 2020-11-20T13:56:47.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74ebfff700 1 -- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec147190 msgr2=0x7f74ec148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.335 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74ebfff700 1 --2- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec147190 0x7f74ec148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.336 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74ebfff700 1 -- 172.21.15.117:0/109380329 >> v1:172.21.15.117:6789/0 conn(0x7f74ec0b8b50 legacy=0x7f74ec14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74ebfff700 1 -- 172.21.15.117:0/109380329 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec147570 con 0x7f74ec09aea0 2020-11-20T13:56:47.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74ebfff700 1 --2- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec09aea0 0x7f74ec061f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f74d8002f30 tx=0x7f74d8006dc0).ready entity=mon.0 client_cookie=b199ac9f79c447f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74eb7fe700 1 -- 172.21.15.117:0/109380329 <== mon.0 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d80096a0 con 0x7f74ec09aea0 2020-11-20T13:56:47.337 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74eb7fe700 1 -- 172.21.15.117:0/109380329 <== mon.0 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d8009800 con 0x7f74ec09aea0 2020-11-20T13:56:47.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7f74eb7fe700 1 -- 172.21.15.117:0/109380329 <== mon.0 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d8009b60 con 0x7f74ec09aea0 2020-11-20T13:56:47.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 -- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec09aea0 msgr2=0x7f74ec061f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 --2- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec09aea0 0x7f74ec061f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f74d8002f30 tx=0x7f74d8006dc0).stop 2020-11-20T13:56:47.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 -- 172.21.15.117:0/109380329 shutdown_connections 2020-11-20T13:56:47.338 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 --2- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec147190 0x7f74ec148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 --2- 172.21.15.117:0/109380329 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec09aea0 0x7f74ec061f40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 -- 172.21.15.117:0/109380329 >> 172.21.15.117:0/109380329 conn(0x7f74ec05c4c0 msgr2=0x7f74ec0b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 -- 172.21.15.117:0/109380329 shutdown_connections 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7f74f321a700 1 -- 172.21.15.117:0/109380329 wait complete. 2020-11-20T13:56:47.339 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.324+0000 7f74f321a700 1 Processor -- start 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.324+0000 7f74f321a700 1 -- start start 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f321a700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec09aea0 0x7f74ec149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f321a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74ec0b8b50 0x7f74ec15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f321a700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec147190 0x7f74ec1435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.340 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f321a700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec09c9b0 con 0x7f74ec147190 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f321a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec15ac40 con 0x7f74ec0b8b50 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f321a700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec159d70 con 0x7f74ec09aea0 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74ebfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74ec0b8b50 0x7f74ec15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74ebfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74ec0b8b50 0x7f74ec15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41954/0 (socket says 172.21.15.117:41954) 2020-11-20T13:56:47.341 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74ebfff700 1 -- 172.21.15.117:0/337947852 learned_addr learned my addr 172.21.15.117:0/337947852 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f17b7700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec147190 0x7f74ec1435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7f74f0fb6700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec09aea0 0x7f74ec149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.342 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74ebfff700 1 -- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec09aea0 msgr2=0x7f74ec149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74ebfff700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec09aea0 0x7f74ec149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74ebfff700 1 -- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec147190 msgr2=0x7f74ec1435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.343 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74ebfff700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec147190 0x7f74ec1435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74ebfff700 1 -- 172.21.15.117:0/337947852 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec15a010 con 0x7f74ec0b8b50 2020-11-20T13:56:47.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74f17b7700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec147190 0x7f74ec1435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74f0fb6700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec09aea0 0x7f74ec149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74ebfff700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74ec0b8b50 0x7f74ec15a750 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f74d8002f30 tx=0x7f74d8015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74e9ffb700 1 -- 172.21.15.117:0/337947852 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d8011b40 con 0x7f74ec0b8b50 2020-11-20T13:56:47.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74e9ffb700 1 -- 172.21.15.117:0/337947852 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d8011ca0 con 0x7f74ec0b8b50 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec15a2b0 con 0x7f74ec0b8b50 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74e9ffb700 1 -- 172.21.15.117:0/337947852 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d801d450 con 0x7f74ec0b8b50 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec0b3440 con 0x7f74ec0b8b50 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.327+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec057050 con 0x7f74ec0b8b50 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7f74e9ffb700 1 -- 172.21.15.117:0/337947852 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f74d8024030 con 0x7f74ec0b8b50 2020-11-20T13:56:47.346 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7f74e9ffb700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f74d40513c0 0x7f74d4053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7f74e9ffb700 1 -- 172.21.15.117:0/337947852 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f74d806a1a0 con 0x7f74ec0b8b50 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7f74f0fb6700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f74d40513c0 0x7f74d4053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7f74f0fb6700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f74d40513c0 0x7f74d4053880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f74e000b000 tx=0x7f74e0008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.331+0000 7f74e9ffb700 1 -- 172.21.15.117:0/337947852 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f74d80423c0 con 0x7f74ec0b8b50 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7fb46ad5c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb4640ae8b0 0x7fb4640aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7fb46ad5c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640b5860 0x7fb464098b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7fb46ad5c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640b4e80 con 0x7fb464099080 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7fb46ad5c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640994f0 con 0x7fb4640ae8b0 2020-11-20T13:56:47.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.321+0000 7fb46ad5c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640b5120 con 0x7fb4640b5860 2020-11-20T13:56:47.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb4692f9700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb464099080 0x7fb464146f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32870/0 (socket says 172.21.15.117:32870) 2020-11-20T13:56:47.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb4692f9700 1 -- 172.21.15.117:0/235587929 learned_addr learned my addr 172.21.15.117:0/235587929 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:47.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb468af8700 1 --2- 172.21.15.117:0/235587929 >> v2:172.21.15.85:3300/0 conn(0x7fb4640ae8b0 0x7fb4640aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb463fff700 1 --2- 172.21.15.117:0/235587929 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640b5860 0x7fb464098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb4637fe700 1 -- 172.21.15.117:0/235587929 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223694634 0 0) 0x7fb4640b4e80 con 0x7fb464099080 2020-11-20T13:56:47.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb4637fe700 1 -- 172.21.15.117:0/235587929 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb444003230 con 0x7fb464099080 2020-11-20T13:56:47.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb463fff700 1 -- 172.21.15.117:0/235587929 >> v2:172.21.15.85:3300/0 conn(0x7fb4640ae8b0 msgr2=0x7fb4640aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:47.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb463fff700 1 --2- 172.21.15.117:0/235587929 >> v2:172.21.15.85:3300/0 conn(0x7fb4640ae8b0 0x7fb4640aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb463fff700 1 -- 172.21.15.117:0/235587929 >> v1:172.21.15.117:6789/0 conn(0x7fb464099080 legacy=0x7fb464146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb463fff700 1 -- 172.21.15.117:0/235587929 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640b53c0 con 0x7fb4640b5860 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb463fff700 1 --2- 172.21.15.117:0/235587929 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640b5860 0x7fb464098b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb454002f30 tx=0x7fb454006dc0).ready entity=mon.2 client_cookie=b3901b07257f2a66 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb4637fe700 1 -- 172.21.15.117:0/235587929 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4540096a0 con 0x7fb4640b5860 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb4637fe700 1 -- 172.21.15.117:0/235587929 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454009800 con 0x7fb4640b5860 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb4637fe700 1 -- 172.21.15.117:0/235587929 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb454009b60 con 0x7fb4640b5860 2020-11-20T13:56:47.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb46ad5c700 1 -- 172.21.15.117:0/235587929 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640b5860 msgr2=0x7fb464098b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.322+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/235587929 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640b5860 0x7fb464098b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb454002f30 tx=0x7fb454006dc0).stop 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7fb46ad5c700 1 -- 172.21.15.117:0/235587929 shutdown_connections 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/235587929 >> v2:172.21.15.85:3300/0 conn(0x7fb4640ae8b0 0x7fb4640aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/235587929 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640b5860 0x7fb464098b90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7fb46ad5c700 1 -- 172.21.15.117:0/235587929 >> 172.21.15.117:0/235587929 conn(0x7fb46405b9f0 msgr2=0x7fb46405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7fb46ad5c700 1 -- 172.21.15.117:0/235587929 shutdown_connections 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.323+0000 7fb46ad5c700 1 -- 172.21.15.117:0/235587929 wait complete. 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.324+0000 7fb46ad5c700 1 Processor -- start 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.324+0000 7fb46ad5c700 1 -- start start 2020-11-20T13:56:47.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb46ad5c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb464099080 0x7fb46413ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb46ad5c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640ae8b0 0x7fb4640659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb46ad5c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640b5860 0x7fb46413a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.354 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb46ad5c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464066360 con 0x7fb464099080 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb46ad5c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464066600 con 0x7fb4640b5860 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb46ad5c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640668a0 con 0x7fb4640ae8b0 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb4692f9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640b5860 0x7fb46413a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb4692f9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640b5860 0x7fb46413a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41952/0 (socket says 172.21.15.117:41952) 2020-11-20T13:56:47.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb4692f9700 1 -- 172.21.15.117:0/899432950 learned_addr learned my addr 172.21.15.117:0/899432950 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb468af8700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb464099080 0x7fb46413ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.325+0000 7fb463fff700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640ae8b0 0x7fb4640659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb4692f9700 1 -- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640ae8b0 msgr2=0x7fb4640659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb4692f9700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640ae8b0 0x7fb4640659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb4692f9700 1 -- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb464099080 msgr2=0x7fb46413ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:47.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb4692f9700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb464099080 0x7fb46413ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:47.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb4692f9700 1 -- 172.21.15.117:0/899432950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640672d0 con 0x7fb4640b5860 2020-11-20T13:56:47.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb468af8700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb464099080 0x7fb46413ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:47.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb4692f9700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640b5860 0x7fb46413a9b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fb44c0010d0 tx=0x7fb44c0079b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb461ffb700 1 -- 172.21.15.117:0/899432950 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb44c0072e0 con 0x7fb4640b5860 2020-11-20T13:56:47.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb461ffb700 1 -- 172.21.15.117:0/899432950 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c007440 con 0x7fb4640b5860 2020-11-20T13:56:47.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.326+0000 7fb461ffb700 1 -- 172.21.15.117:0/899432950 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb44c0057d0 con 0x7fb4640b5860 2020-11-20T13:56:47.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.327+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb464065ed0 con 0x7fb4640b5860 2020-11-20T13:56:47.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.327+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46413b0e0 con 0x7fb4640b5860 2020-11-20T13:56:47.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7fb461ffb700 1 -- 172.21.15.117:0/899432950 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fb44c010070 con 0x7fb4640b5860 2020-11-20T13:56:47.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464066180 con 0x7fb4640b5860 2020-11-20T13:56:47.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7fb461ffb700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb448051210 0x7fb4480536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:47.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7fb461ffb700 1 -- 172.21.15.117:0/899432950 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6420+0+0 (secure 0 0 0) 0x7fb44c0688d0 con 0x7fb4640b5860 2020-11-20T13:56:47.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7fb468af8700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb448051210 0x7fb4480536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:47.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.328+0000 7fb468af8700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb448051210 0x7fb4480536d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb45c00b000 tx=0x7fb45c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:47.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.335+0000 7fb461ffb700 1 -- 172.21.15.117:0/899432950 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7fb44c040af0 con 0x7fb4640b5860 2020-11-20T13:56:47.550 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.549+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f74ec0b68a0 con 0x7f74ec0b8b50 2020-11-20T13:56:47.552 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:47.551+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fb464057050 con 0x7fb4640b5860 2020-11-20T13:56:48.206 INFO:tasks.ceph.mon.b.smithi085.stderr:2020-11-20T13:56:48.204+0000 7f2a23f2f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T13:56:48.208 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.207+0000 7f74e9ffb700 1 -- 172.21.15.117:0/337947852 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 4996 v11) v1 ==== 95+0+0 (secure 0 0 0) 0x7f74d8022020 con 0x7f74ec0b8b50 2020-11-20T13:56:48.209 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.207+0000 7fb461ffb700 1 -- 172.21.15.117:0/899432950 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 4827 v11) v1 ==== 95+0+0 (secure 0 0 0) 0x7fb44c044110 con 0x7fb4640b5860 2020-11-20T13:56:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f74d40513c0 msgr2=0x7f74d4053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.211 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7f74f321a700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f74d40513c0 0x7f74d4053880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f74e000b000 tx=0x7f74e0008040).stop 2020-11-20T13:56:48.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74ec0b8b50 msgr2=0x7f74ec15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7f74f321a700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74ec0b8b50 0x7f74ec15a750 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f74d8002f30 tx=0x7f74d8015000).stop 2020-11-20T13:56:48.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 shutdown_connections 2020-11-20T13:56:48.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f74d40513c0 0x7f74d4053880 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.212 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f74ec09aea0 0x7f74ec149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.213 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f74ec147190 0x7f74ec1435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.213 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 --2- 172.21.15.117:0/337947852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74ec0b8b50 0x7f74ec15a750 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.214 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 >> 172.21.15.117:0/337947852 conn(0x7f74ec05c4c0 msgr2=0x7f74ec05f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.214 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 shutdown_connections 2020-11-20T13:56:48.214 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7f74f321a700 1 -- 172.21.15.117:0/337947852 wait complete. 2020-11-20T13:56:48.214 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb448051210 msgr2=0x7fb4480536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.214 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb448051210 0x7fb4480536d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fb45c00b000 tx=0x7fb45c008040).stop 2020-11-20T13:56:48.215 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640b5860 msgr2=0x7fb46413a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.215 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.209+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640b5860 0x7fb46413a9b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fb44c0010d0 tx=0x7fb44c0079b0).stop 2020-11-20T13:56:48.215 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 shutdown_connections 2020-11-20T13:56:48.215 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fb448051210 0x7fb4480536d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.216 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4640b5860 0x7fb46413a9b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.216 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb464099080 0x7fb46413ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.216 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 --2- 172.21.15.117:0/899432950 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb4640ae8b0 0x7fb4640659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.216 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 >> 172.21.15.117:0/899432950 conn(0x7fb46405b9f0 msgr2=0x7fb4640b9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.216 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 shutdown_connections 2020-11-20T13:56:48.217 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.210+0000 7fb46ad5c700 1 -- 172.21.15.117:0/899432950 wait complete. 2020-11-20T13:56:48.218 INFO:teuthology.orchestra.run.smithi117.stderr:failed mds gid 4827 2020-11-20T13:56:48.218 INFO:teuthology.orchestra.run.smithi117.stderr:failed mds gid 4996 2020-11-20T13:56:48.225 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T13:56:48.226 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T13:56:48.226 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T13:56:48.227 INFO:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T13:56:48.228 INFO:tasks.ceph.mds.a:Started 2020-11-20T13:56:48.229 DEBUG:teuthology.parallel:result is None 2020-11-20T13:56:48.230 INFO:tasks.ceph.mds.b:Started 2020-11-20T13:56:48.230 DEBUG:teuthology.parallel:result is None 2020-11-20T13:56:48.230 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:48.307 INFO:tasks.ceph.mds.a.smithi117.stdout:starting mds.a at 2020-11-20T13:56:48.311 INFO:tasks.ceph.mds.b.smithi085.stdout:starting mds.b at 2020-11-20T13:56:48.344 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.343+0000 7f2c15293700 1 Processor -- start 2020-11-20T13:56:48.345 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.343+0000 7f2c15293700 1 -- start start 2020-11-20T13:56:48.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.343+0000 7f2c15293700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c10141480 0x7f2c10147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c15293700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c1014a830 0x7f2c10060630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c15293700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c100b6780 con 0x7f2c1005cd80 2020-11-20T13:56:48.347 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c15293700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c10147850 con 0x7f2c10141480 2020-11-20T13:56:48.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c15293700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10147af0 con 0x7f2c1014a830 2020-11-20T13:56:48.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c10141480 0x7f2c10147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c10141480 0x7f2c10147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41976/0 (socket says 172.21.15.117:41976) 2020-11-20T13:56:48.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c0effd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2c1005cd80 0x7f2c10140d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32902/0 (socket says 172.21.15.117:32902) 2020-11-20T13:56:48.348 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 -- 172.21.15.117:0/855826790 learned_addr learned my addr 172.21.15.117:0/855826790 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:48.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c0f7fe700 1 --2- 172.21.15.117:0/855826790 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c1014a830 0x7f2c10060630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 -- 172.21.15.117:0/855826790 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c1014a830 msgr2=0x7f2c10060630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:48.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 --2- 172.21.15.117:0/855826790 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c1014a830 0x7f2c10060630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 -- 172.21.15.117:0/855826790 >> v1:172.21.15.117:6789/0 conn(0x7f2c1005cd80 legacy=0x7f2c10140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.349 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 -- 172.21.15.117:0/855826790 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c10147d90 con 0x7f2c10141480 2020-11-20T13:56:48.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c07fff700 1 --2- 172.21.15.117:0/855826790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c10141480 0x7f2c10147220 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f2c000010d0 tx=0x7f2bf8006e80).ready entity=mon.1 client_cookie=281263257e0d85be server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c0e7fc700 1 -- 172.21.15.117:0/855826790 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf80096b0 con 0x7f2c10141480 2020-11-20T13:56:48.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c0e7fc700 1 -- 172.21.15.117:0/855826790 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf8009810 con 0x7f2c10141480 2020-11-20T13:56:48.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c0e7fc700 1 -- 172.21.15.117:0/855826790 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf8009980 con 0x7f2c10141480 2020-11-20T13:56:48.350 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c15293700 1 -- 172.21.15.117:0/855826790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c10141480 msgr2=0x7f2c10147220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.344+0000 7f2c15293700 1 --2- 172.21.15.117:0/855826790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c10141480 0x7f2c10147220 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f2c000010d0 tx=0x7f2bf8006e80).stop 2020-11-20T13:56:48.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.345+0000 7f2c15293700 1 -- 172.21.15.117:0/855826790 shutdown_connections 2020-11-20T13:56:48.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.345+0000 7f2c15293700 1 --2- 172.21.15.117:0/855826790 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c1014a830 0x7f2c10060630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.345+0000 7f2c15293700 1 --2- 172.21.15.117:0/855826790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c10141480 0x7f2c10147220 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.351 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.345+0000 7f2c15293700 1 -- 172.21.15.117:0/855826790 >> 172.21.15.117:0/855826790 conn(0x7f2c100b7480 msgr2=0x7f2c10062f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.345+0000 7f2c15293700 1 -- 172.21.15.117:0/855826790 shutdown_connections 2020-11-20T13:56:48.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.345+0000 7f2c15293700 1 -- 172.21.15.117:0/855826790 wait complete. 2020-11-20T13:56:48.352 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 Processor -- start 2020-11-20T13:56:48.353 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 -- start start 2020-11-20T13:56:48.355 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c1005cd80 0x7f2c10143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c10141480 0x7f2c100b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c1014a830 0x7f2c10146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10142720 con 0x7f2c1014a830 2020-11-20T13:56:48.356 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101429c0 con 0x7f2c1005cd80 2020-11-20T13:56:48.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c15293700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10142c60 con 0x7f2c10141480 2020-11-20T13:56:48.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.346+0000 7f2c0effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c1005cd80 0x7f2c10143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c1005cd80 0x7f2c10143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41980/0 (socket says 172.21.15.117:41980) 2020-11-20T13:56:48.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 -- 172.21.15.117:0/3389205627 learned_addr learned my addr 172.21.15.117:0/3389205627 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:48.357 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c07fff700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c10141480 0x7f2c100b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0f7fe700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c1014a830 0x7f2c10146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 -- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c10141480 msgr2=0x7f2c100b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c10141480 0x7f2c100b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 -- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c1014a830 msgr2=0x7f2c10146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.358 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c1014a830 0x7f2c10146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 -- 172.21.15.117:0/3389205627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1009ef10 con 0x7f2c1005cd80 2020-11-20T13:56:48.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0effd700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c1005cd80 0x7f2c10143100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f2c00001190 tx=0x7f2c00007b40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0cff9700 1 -- 172.21.15.117:0/3389205627 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c00007320 con 0x7f2c1005cd80 2020-11-20T13:56:48.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0cff9700 1 -- 172.21.15.117:0/3389205627 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c00005520 con 0x7f2c1005cd80 2020-11-20T13:56:48.359 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c101423a0 con 0x7f2c1005cd80 2020-11-20T13:56:48.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c0cff9700 1 -- 172.21.15.117:0/3389205627 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c00005880 con 0x7f2c1005cd80 2020-11-20T13:56:48.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.347+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c10155bc0 con 0x7f2c1005cd80 2020-11-20T13:56:48.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.348+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c10142530 con 0x7f2c1005cd80 2020-11-20T13:56:48.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.348+0000 7f2c0cff9700 1 -- 172.21.15.117:0/3389205627 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f2c00010030 con 0x7f2c1005cd80 2020-11-20T13:56:48.360 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.349+0000 7f2c0cff9700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2bf4051210 0x7f2bf40536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:48.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.349+0000 7f2c0cff9700 1 -- 172.21.15.117:0/3389205627 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f2c00068f00 con 0x7f2c1005cd80 2020-11-20T13:56:48.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.349+0000 7f2c07fff700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2bf4051210 0x7f2bf40536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:48.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.349+0000 7f2c07fff700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2bf4051210 0x7f2bf40536d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f2bf801b000 tx=0x7f2bf8016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:48.361 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.352+0000 7f2c0cff9700 1 -- 172.21.15.117:0/3389205627 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f2c00041030 con 0x7f2c1005cd80 2020-11-20T13:56:48.553 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.552+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c10057050 con 0x7f2c1005cd80 2020-11-20T13:56:48.554 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.552+0000 7f2c0cff9700 1 -- 172.21.15.117:0/3389205627 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1425 (secure 0 0 0) 0x7f2c00044650 con 0x7f2c1005cd80 2020-11-20T13:56:48.554 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:48.554 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:48.205615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2bf4051210 msgr2=0x7f2bf40536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2bf4051210 0x7f2bf40536d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f2bf801b000 tx=0x7f2bf8016000).stop 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c1005cd80 msgr2=0x7f2c10143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c1005cd80 0x7f2c10143100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f2c00001190 tx=0x7f2c00007b40).stop 2020-11-20T13:56:48.556 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 shutdown_connections 2020-11-20T13:56:48.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f2bf4051210 0x7f2bf40536d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f2c1014a830 0x7f2c10146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c1005cd80 0x7f2c10143100 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.557 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 --2- 172.21.15.117:0/3389205627 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f2c10141480 0x7f2c100b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:48.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 >> 172.21.15.117:0/3389205627 conn(0x7f2c100b7480 msgr2=0x7f2c100b7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:48.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 shutdown_connections 2020-11-20T13:56:48.558 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:48.554+0000 7f2c15293700 1 -- 172.21.15.117:0/3389205627 wait complete. 2020-11-20T13:56:48.558 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 11 2020-11-20T13:56:48.767 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:33.416903+0000', 'modified': '2020-11-20T13:56:48.205615+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:48.768 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2020-11-20T13:56:49.769 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:49.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.886+0000 7f127942e700 1 Processor -- start 2020-11-20T13:56:49.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.886+0000 7f127942e700 1 -- start start 2020-11-20T13:56:49.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f127942e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1274145ae0 0x7f127409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:49.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f127942e700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12740b5100 con 0x7f127414eff0 2020-11-20T13:56:49.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f127942e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127414fae0 con 0x7f1274097f80 2020-11-20T13:56:49.894 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f127942e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1274098490 con 0x7f1274145ae0 2020-11-20T13:56:49.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f126bfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1274097f80 0x7f12741453d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32934/0 (socket says 172.21.15.117:32934) 2020-11-20T13:56:49.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f126bfff700 1 -- 172.21.15.117:0/3679235786 learned_addr learned my addr 172.21.15.117:0/3679235786 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:49.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12737fe700 1 --2- 172.21.15.117:0/3679235786 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1274145ae0 0x7f127409b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:49.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12727fc700 1 -- 172.21.15.117:0/3679235786 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081696357 0 0) 0x7f127414fae0 con 0x7f1274097f80 2020-11-20T13:56:49.895 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12727fc700 1 -- 172.21.15.117:0/3679235786 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1254003230 con 0x7f1274097f80 2020-11-20T13:56:49.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12727fc700 1 -- 172.21.15.117:0/3679235786 <== mon.0 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709476150 0 0) 0x7f12740b5100 con 0x7f127414eff0 2020-11-20T13:56:49.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12727fc700 1 -- 172.21.15.117:0/3679235786 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f127414fae0 con 0x7f127414eff0 2020-11-20T13:56:49.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12737fe700 1 -- 172.21.15.117:0/3679235786 >> v1:172.21.15.117:6789/0 conn(0x7f1274097f80 legacy=0x7f12741453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:49.896 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12737fe700 1 -- 172.21.15.117:0/3679235786 >> v1:172.21.15.85:6789/0 conn(0x7f127414eff0 legacy=0x7f127414f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:49.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12737fe700 1 -- 172.21.15.117:0/3679235786 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274098730 con 0x7f1274145ae0 2020-11-20T13:56:49.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.887+0000 7f12737fe700 1 --2- 172.21.15.117:0/3679235786 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1274145ae0 0x7f127409b6e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f126c0032b0 tx=0x7f126c0091a0).ready entity=mon.2 client_cookie=ab605cc2595514b4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:49.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f12727fc700 1 -- 172.21.15.117:0/3679235786 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f126c00b940 con 0x7f1274145ae0 2020-11-20T13:56:49.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f12727fc700 1 -- 172.21.15.117:0/3679235786 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c00baa0 con 0x7f1274145ae0 2020-11-20T13:56:49.897 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f12727fc700 1 -- 172.21.15.117:0/3679235786 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f126c00bc10 con 0x7f1274145ae0 2020-11-20T13:56:49.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f127942e700 1 -- 172.21.15.117:0/3679235786 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1274145ae0 msgr2=0x7f127409b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:49.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f127942e700 1 --2- 172.21.15.117:0/3679235786 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1274145ae0 0x7f127409b6e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f126c0032b0 tx=0x7f126c0091a0).stop 2020-11-20T13:56:49.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f127942e700 1 -- 172.21.15.117:0/3679235786 shutdown_connections 2020-11-20T13:56:49.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f127942e700 1 --2- 172.21.15.117:0/3679235786 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f1274145ae0 0x7f127409b6e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:49.898 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f127942e700 1 -- 172.21.15.117:0/3679235786 >> 172.21.15.117:0/3679235786 conn(0x7f127405b9f0 msgr2=0x7f127405be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:49.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f127942e700 1 -- 172.21.15.117:0/3679235786 shutdown_connections 2020-11-20T13:56:49.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.888+0000 7f127942e700 1 -- 172.21.15.117:0/3679235786 wait complete. 2020-11-20T13:56:49.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.889+0000 7f127942e700 1 Processor -- start 2020-11-20T13:56:49.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.889+0000 7f127942e700 1 -- start start 2020-11-20T13:56:49.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f127942e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1274097f80 0x7f12740b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:49.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f127942e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1274145ae0 0x7f12740b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:49.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f127942e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f127414eff0 0x7f1274148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:49.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f127942e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127415b270 con 0x7f1274145ae0 2020-11-20T13:56:49.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f127942e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127415b510 con 0x7f1274097f80 2020-11-20T13:56:49.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f127942e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127415b7b0 con 0x7f127414eff0 2020-11-20T13:56:49.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f1272ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1274097f80 0x7f12740b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:49.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f1272ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1274097f80 0x7f12740b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42008/0 (socket says 172.21.15.117:42008) 2020-11-20T13:56:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f1272ffd700 1 -- 172.21.15.117:0/2929472817 learned_addr learned my addr 172.21.15.117:0/2929472817 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f12737fe700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f127414eff0 0x7f1274148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.890+0000 7f126bfff700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1274145ae0 0x7f12740b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1272ffd700 1 -- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f127414eff0 msgr2=0x7f1274148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1272ffd700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f127414eff0 0x7f1274148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1272ffd700 1 -- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1274145ae0 msgr2=0x7f12740b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1272ffd700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1274145ae0 0x7f12740b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1272ffd700 1 -- 172.21.15.117:0/2929472817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12740b7a90 con 0x7f1274097f80 2020-11-20T13:56:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1272ffd700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1274097f80 0x7f12740b9a80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f12640010d0 tx=0x7f12640079e0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1270ff9700 1 -- 172.21.15.117:0/2929472817 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f126400e4a0 con 0x7f1274097f80 2020-11-20T13:56:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1270ff9700 1 -- 172.21.15.117:0/2929472817 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126400e600 con 0x7f1274097f80 2020-11-20T13:56:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12740b2540 con 0x7f1274097f80 2020-11-20T13:56:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f1270ff9700 1 -- 172.21.15.117:0/2929472817 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f126400e960 con 0x7f1274097f80 2020-11-20T13:56:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.891+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1274148580 con 0x7f1274097f80 2020-11-20T13:56:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.892+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1274057050 con 0x7f1274097f80 2020-11-20T13:56:49.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.892+0000 7f1270ff9700 1 -- 172.21.15.117:0/2929472817 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f1264019030 con 0x7f1274097f80 2020-11-20T13:56:49.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.892+0000 7f1270ff9700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1258051320 0x7f12580537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:49.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.892+0000 7f1270ff9700 1 -- 172.21.15.117:0/2929472817 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f12640686a0 con 0x7f1274097f80 2020-11-20T13:56:49.907 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.892+0000 7f126bfff700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1258051320 0x7f12580537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.893+0000 7f126bfff700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1258051320 0x7f12580537e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f125c00b000 tx=0x7f125c009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:49.908 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:49.896+0000 7f1270ff9700 1 -- 172.21.15.117:0/2929472817 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f1264040870 con 0x7f1274097f80 2020-11-20T13:56:50.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.098+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12740b6cd0 con 0x7f1274097f80 2020-11-20T13:56:50.100 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.098+0000 7f1270ff9700 1 -- 172.21.15.117:0/2929472817 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f1264043e90 con 0x7f1274097f80 2020-11-20T13:56:50.100 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:50.101 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5022,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/1931748750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":1931748750},{"type":"v1","addr":"172.21.15.85:6833","nonce":1931748750}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:49.218656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"a","rank":0,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3369439478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3369439478},{"type":"v1","addr":"172.21.15.117:6835","nonce":3369439478}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:50.101 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1258051320 msgr2=0x7f12580537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1258051320 0x7f12580537e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f125c00b000 tx=0x7f125c009040).stop 2020-11-20T13:56:50.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1274097f80 msgr2=0x7f12740b9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:50.102 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1274097f80 0x7f12740b9a80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f12640010d0 tx=0x7f12640079e0).stop 2020-11-20T13:56:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 shutdown_connections 2020-11-20T13:56:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f1258051320 0x7f12580537e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f127414eff0 0x7f1274148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1274097f80 0x7f12740b9a80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 --2- 172.21.15.117:0/2929472817 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f1274145ae0 0x7f12740b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 >> 172.21.15.117:0/2929472817 conn(0x7f127405b9f0 msgr2=0x7f1274144e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 shutdown_connections 2020-11-20T13:56:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:50.100+0000 7f127942e700 1 -- 172.21.15.117:0/2929472817 wait complete. 2020-11-20T13:56:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 13 2020-11-20T13:56:50.313 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:33.416903+0000', 'modified': '2020-11-20T13:56:49.218656+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5242': {'gid': 5242, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/3369439478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3369439478}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3369439478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:50.314 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5242:up:replay 2020-11-20T13:56:51.315 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:51.432 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.431+0000 7f3722a96700 1 Processor -- start 2020-11-20T13:56:51.433 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.431+0000 7f3722a96700 1 -- start start 2020-11-20T13:56:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3722a96700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f371c098e30 0x7f371c09b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3722a96700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f371c0ae6f0 0x7f371c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3722a96700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c0b5510 con 0x7f371c09b700 2020-11-20T13:56:51.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3722a96700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c0b57b0 con 0x7f371c0ae6f0 2020-11-20T13:56:51.436 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3722a96700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c0b5a50 con 0x7f371c098e30 2020-11-20T13:56:51.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3720832700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f371c098e30 0x7f371c09b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f371bfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f371c09b700 0x7f371c1438e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32946/0 (socket says 172.21.15.117:32946) 2020-11-20T13:56:51.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3720832700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f371c098e30 0x7f371c09b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42018/0 (socket says 172.21.15.117:42018) 2020-11-20T13:56:51.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f371bfff700 1 -- 172.21.15.117:0/868135450 learned_addr learned my addr 172.21.15.117:0/868135450 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:51.437 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3721033700 1 --2- 172.21.15.117:0/868135450 >> v2:172.21.15.85:3300/0 conn(0x7f371c0ae6f0 0x7f371c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3720832700 1 -- 172.21.15.117:0/868135450 >> v2:172.21.15.85:3300/0 conn(0x7f371c0ae6f0 msgr2=0x7f371c0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:51.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3720832700 1 --2- 172.21.15.117:0/868135450 >> v2:172.21.15.85:3300/0 conn(0x7f371c0ae6f0 0x7f371c0aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3720832700 1 -- 172.21.15.117:0/868135450 >> v1:172.21.15.117:6789/0 conn(0x7f371c09b700 legacy=0x7f371c1438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f3720832700 1 -- 172.21.15.117:0/868135450 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c099210 con 0x7f371c098e30 2020-11-20T13:56:51.438 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.432+0000 7f371b7fe700 1 -- 172.21.15.117:0/868135450 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412575753 0 0) 0x7f371c0b5510 con 0x7f371c09b700 2020-11-20T13:56:51.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3720832700 1 --2- 172.21.15.117:0/868135450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c098e30 0x7f371c09b210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f370c002f30 tx=0x7f370c006dc0).ready entity=mon.1 client_cookie=2f4d31c02cc65551 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f371b7fe700 1 -- 172.21.15.117:0/868135450 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370c0097f0 con 0x7f371c098e30 2020-11-20T13:56:51.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f371b7fe700 1 -- 172.21.15.117:0/868135450 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c009950 con 0x7f371c098e30 2020-11-20T13:56:51.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f371b7fe700 1 -- 172.21.15.117:0/868135450 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370c009cb0 con 0x7f371c098e30 2020-11-20T13:56:51.439 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 -- 172.21.15.117:0/868135450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c098e30 msgr2=0x7f371c09b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 --2- 172.21.15.117:0/868135450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c098e30 0x7f371c09b210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f370c002f30 tx=0x7f370c006dc0).stop 2020-11-20T13:56:51.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 -- 172.21.15.117:0/868135450 shutdown_connections 2020-11-20T13:56:51.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 --2- 172.21.15.117:0/868135450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c098e30 0x7f371c09b210 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 --2- 172.21.15.117:0/868135450 >> v2:172.21.15.85:3300/0 conn(0x7f371c0ae6f0 0x7f371c0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.440 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 -- 172.21.15.117:0/868135450 >> 172.21.15.117:0/868135450 conn(0x7f371c05b9f0 msgr2=0x7f371c05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 -- 172.21.15.117:0/868135450 shutdown_connections 2020-11-20T13:56:51.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.433+0000 7f3722a96700 1 -- 172.21.15.117:0/868135450 wait complete. 2020-11-20T13:56:51.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.434+0000 7f3722a96700 1 Processor -- start 2020-11-20T13:56:51.441 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.434+0000 7f3722a96700 1 -- start start 2020-11-20T13:56:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f3722a96700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f371c098e30 0x7f371c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f3722a96700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c09b700 0x7f371c154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f3722a96700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f371c0ae6f0 0x7f371c1438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.444 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f3722a96700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c154ef0 con 0x7f371c098e30 2020-11-20T13:56:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f3722a96700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0b2d90 con 0x7f371c09b700 2020-11-20T13:56:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f3722a96700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0b3030 con 0x7f371c0ae6f0 2020-11-20T13:56:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f371bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c09b700 0x7f371c154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f371bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c09b700 0x7f371c154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42026/0 (socket says 172.21.15.117:42026) 2020-11-20T13:56:51.445 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f371bfff700 1 -- 172.21.15.117:0/299282333 learned_addr learned my addr 172.21.15.117:0/299282333 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.435+0000 7f3720832700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f371c098e30 0x7f371c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f3721033700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f371c0ae6f0 0x7f371c1438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f371bfff700 1 -- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f371c0ae6f0 msgr2=0x7f371c1438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f371bfff700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f371c0ae6f0 0x7f371c1438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.446 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f371bfff700 1 -- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f371c098e30 msgr2=0x7f371c0b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f371bfff700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f371c098e30 0x7f371c0b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f371bfff700 1 -- 172.21.15.117:0/299282333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c143dc0 con 0x7f371c09b700 2020-11-20T13:56:51.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f3720832700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f371c098e30 0x7f371c0b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:51.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f371bfff700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c09b700 0x7f371c154a00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f3710000b80 tx=0x7f3710007a30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.447 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f3719ffb700 1 -- 172.21.15.117:0/299282333 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37100072e0 con 0x7f371c09b700 2020-11-20T13:56:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f3719ffb700 1 -- 172.21.15.117:0/299282333 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3710007440 con 0x7f371c09b700 2020-11-20T13:56:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c145300 con 0x7f371c09b700 2020-11-20T13:56:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f3719ffb700 1 -- 172.21.15.117:0/299282333 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37100057d0 con 0x7f371c09b700 2020-11-20T13:56:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.436+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c142ca0 con 0x7f371c09b700 2020-11-20T13:56:51.448 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.437+0000 7f3719ffb700 1 -- 172.21.15.117:0/299282333 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3710010070 con 0x7f371c09b700 2020-11-20T13:56:51.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.437+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c0b31f0 con 0x7f371c09b700 2020-11-20T13:56:51.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.437+0000 7f3719ffb700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36fc051210 0x7f36fc0536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:51.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.437+0000 7f3719ffb700 1 -- 172.21.15.117:0/299282333 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f37100688a0 con 0x7f371c09b700 2020-11-20T13:56:51.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.437+0000 7f3720832700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36fc051210 0x7f36fc0536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:51.449 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.437+0000 7f3720832700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36fc051210 0x7f36fc0536d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f370c016000 tx=0x7f370c015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:51.450 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.441+0000 7f3719ffb700 1 -- 172.21.15.117:0/299282333 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3710040a70 con 0x7f371c09b700 2020-11-20T13:56:51.641 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.640+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f371c057050 con 0x7f371c09b700 2020-11-20T13:56:51.642 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.640+0000 7f3719ffb700 1 -- 172.21.15.117:0/299282333 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f3710044090 con 0x7f371c09b700 2020-11-20T13:56:51.642 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:51.642 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5022,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/1931748750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":1931748750},{"type":"v1","addr":"172.21.15.85:6833","nonce":1931748750}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:51.222716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"a","rank":0,"incarnation":13,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.117:6835/3369439478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3369439478},{"type":"v1","addr":"172.21.15.117:6835","nonce":3369439478}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:51.643 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36fc051210 msgr2=0x7f36fc0536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36fc051210 0x7f36fc0536d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f370c016000 tx=0x7f370c015000).stop 2020-11-20T13:56:51.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c09b700 msgr2=0x7f371c154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:51.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c09b700 0x7f371c154a00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f3710000b80 tx=0x7f3710007a30).stop 2020-11-20T13:56:51.644 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 shutdown_connections 2020-11-20T13:56:51.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f36fc051210 0x7f36fc0536d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f371c0ae6f0 0x7f371c1438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f371c098e30 0x7f371c0b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 --2- 172.21.15.117:0/299282333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371c09b700 0x7f371c154a00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:51.645 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 >> 172.21.15.117:0/299282333 conn(0x7f371c05b9f0 msgr2=0x7f371c0b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:51.646 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 shutdown_connections 2020-11-20T13:56:51.646 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:51.642+0000 7f3722a96700 1 -- 172.21.15.117:0/299282333 wait complete. 2020-11-20T13:56:51.646 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 15 2020-11-20T13:56:51.856 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:33.416903+0000', 'modified': '2020-11-20T13:56:51.222716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5242': {'gid': 5242, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.117:6835/3369439478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3369439478}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3369439478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:51.856 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5242:up:rejoin 2020-11-20T13:56:52.103 INFO:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:52.106 INFO:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T13:56:52.858 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:52.973 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.972+0000 7f56a302e700 1 Processor -- start 2020-11-20T13:56:52.974 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.972+0000 7f56a302e700 1 -- start start 2020-11-20T13:56:52.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a302e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.976 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a302e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f569c14b7a0 0x7f569c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a302e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c1473d0 con 0x7f569c0b6df0 2020-11-20T13:56:52.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a302e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c14adc0 con 0x7f569c14b7a0 2020-11-20T13:56:52.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a302e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c14b060 con 0x7f569c146ff0 2020-11-20T13:56:52.977 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f569c14b7a0 0x7f569c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f569bfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f569c0b6df0 0x7f569c0b5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32962/0 (socket says 172.21.15.117:32962) 2020-11-20T13:56:52.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f569c14b7a0 0x7f569c0b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42034/0 (socket says 172.21.15.117:42034) 2020-11-20T13:56:52.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 -- 172.21.15.117:0/587867554 learned_addr learned my addr 172.21.15.117:0/587867554 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:52.978 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a0dca700 1 --2- 172.21.15.117:0/587867554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 -- 172.21.15.117:0/587867554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 msgr2=0x7f569c0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:52.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 --2- 172.21.15.117:0/587867554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 -- 172.21.15.117:0/587867554 >> v1:172.21.15.117:6789/0 conn(0x7f569c0b6df0 legacy=0x7f569c0b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.979 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f569b7fe700 1 -- 172.21.15.117:0/587867554 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945045748 0 0) 0x7f569c1473d0 con 0x7f569c0b6df0 2020-11-20T13:56:52.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 -- 172.21.15.117:0/587867554 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c14b300 con 0x7f569c14b7a0 2020-11-20T13:56:52.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a0dca700 1 --2- 172.21.15.117:0/587867554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:52.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a15cb700 1 --2- 172.21.15.117:0/587867554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c14b7a0 0x7f569c0b40f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f568c002f30 tx=0x7f568c006e80).ready entity=mon.1 client_cookie=8c55b0c2ea3b3cdc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.980 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f569b7fe700 1 -- 172.21.15.117:0/587867554 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f568c0097f0 con 0x7f569c14b7a0 2020-11-20T13:56:52.981 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f569b7fe700 1 -- 172.21.15.117:0/587867554 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c009950 con 0x7f569c14b7a0 2020-11-20T13:56:52.981 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f569b7fe700 1 -- 172.21.15.117:0/587867554 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f568c009cb0 con 0x7f569c14b7a0 2020-11-20T13:56:52.981 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a302e700 1 -- 172.21.15.117:0/587867554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c14b7a0 msgr2=0x7f569c0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.981 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.973+0000 7f56a302e700 1 --2- 172.21.15.117:0/587867554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c14b7a0 0x7f569c0b40f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f568c002f30 tx=0x7f568c006e80).stop 2020-11-20T13:56:52.981 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.974+0000 7f56a302e700 1 -- 172.21.15.117:0/587867554 shutdown_connections 2020-11-20T13:56:52.982 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.974+0000 7f56a302e700 1 --2- 172.21.15.117:0/587867554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c14b7a0 0x7f569c0b40f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.982 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.974+0000 7f56a302e700 1 --2- 172.21.15.117:0/587867554 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.982 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.974+0000 7f56a302e700 1 -- 172.21.15.117:0/587867554 >> 172.21.15.117:0/587867554 conn(0x7f569c05b9f0 msgr2=0x7f569c05be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:52.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.974+0000 7f56a302e700 1 -- 172.21.15.117:0/587867554 shutdown_connections 2020-11-20T13:56:52.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.974+0000 7f56a302e700 1 -- 172.21.15.117:0/587867554 wait complete. 2020-11-20T13:56:52.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.975+0000 7f56a302e700 1 Processor -- start 2020-11-20T13:56:52.983 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.975+0000 7f56a302e700 1 -- start start 2020-11-20T13:56:52.985 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a302e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c0b6df0 0x7f569c09b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a302e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a302e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f569c14b7a0 0x7f569c0905b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a302e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c08fe70 con 0x7f569c14b7a0 2020-11-20T13:56:52.986 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a302e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c090110 con 0x7f569c0b6df0 2020-11-20T13:56:52.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a302e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c157360 con 0x7f569c146ff0 2020-11-20T13:56:52.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c0b6df0 0x7f569c09b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.987 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c0b6df0 0x7f569c09b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42038/0 (socket says 172.21.15.117:42038) 2020-11-20T13:56:52.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 -- 172.21.15.117:0/2703550571 learned_addr learned my addr 172.21.15.117:0/2703550571 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:52.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f569bfff700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a15cb700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f569c14b7a0 0x7f569c0905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 -- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 msgr2=0x7f569c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.988 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 -- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f569c14b7a0 msgr2=0x7f569c0905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:52.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f569c14b7a0 0x7f569c0905b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:52.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 -- 172.21.15.117:0/2703550571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0b2950 con 0x7f569c0b6df0 2020-11-20T13:56:52.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a0dca700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c0b6df0 0x7f569c09b8b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f5694007000 tx=0x7f5694008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.989 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f5699ffb700 1 -- 172.21.15.117:0/2703550571 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5694007950 con 0x7f569c0b6df0 2020-11-20T13:56:52.990 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f5699ffb700 1 -- 172.21.15.117:0/2703550571 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5694007ab0 con 0x7f569c0b6df0 2020-11-20T13:56:52.990 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c0b0f60 con 0x7f569c0b6df0 2020-11-20T13:56:52.990 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.976+0000 7f5699ffb700 1 -- 172.21.15.117:0/2703550571 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5694007c20 con 0x7f569c0b6df0 2020-11-20T13:56:52.990 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.977+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c156fd0 con 0x7f569c0b6df0 2020-11-20T13:56:52.990 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.977+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c057050 con 0x7f569c0b6df0 2020-11-20T13:56:52.991 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.978+0000 7f5699ffb700 1 -- 172.21.15.117:0/2703550571 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f5694011410 con 0x7f569c0b6df0 2020-11-20T13:56:52.991 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.978+0000 7f5699ffb700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f567c0513c0 0x7f567c053880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:52.991 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.978+0000 7f5699ffb700 1 -- 172.21.15.117:0/2703550571 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f56940692e0 con 0x7f569c0b6df0 2020-11-20T13:56:52.991 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.978+0000 7f569bfff700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f567c0513c0 0x7f567c053880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:52.991 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.978+0000 7f569bfff700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f567c0513c0 0x7f567c053880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f569000b000 tx=0x7f56900070e0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:52.992 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:52.981+0000 7f5699ffb700 1 -- 172.21.15.117:0/2703550571 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f5694041450 con 0x7f569c0b6df0 2020-11-20T13:56:53.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.183+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f569c09f7d0 con 0x7f569c0b6df0 2020-11-20T13:56:53.185 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.183+0000 7f5699ffb700 1 -- 172.21.15.117:0/2703550571 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f5694044a70 con 0x7f569c0b6df0 2020-11-20T13:56:53.185 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:53.185 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5022,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/1931748750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":1931748750},{"type":"v1","addr":"172.21.15.85:6833","nonce":1931748750}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:52.226497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6835/3369439478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3369439478},{"type":"v1","addr":"172.21.15.117:6835","nonce":3369439478}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:53.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f567c0513c0 msgr2=0x7f567c053880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f567c0513c0 0x7f567c053880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f569000b000 tx=0x7f56900070e0).stop 2020-11-20T13:56:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c0b6df0 msgr2=0x7f569c09b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c0b6df0 0x7f569c09b8b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f5694007000 tx=0x7f5694008040).stop 2020-11-20T13:56:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 shutdown_connections 2020-11-20T13:56:53.187 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f567c0513c0 0x7f567c053880 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f569c14b7a0 0x7f569c0905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f569c0b6df0 0x7f569c09b8b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 --2- 172.21.15.117:0/2703550571 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f569c146ff0 0x7f569c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 >> 172.21.15.117:0/2703550571 conn(0x7f569c05b9f0 msgr2=0x7f569c0b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 shutdown_connections 2020-11-20T13:56:53.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.185+0000 7f56a302e700 1 -- 172.21.15.117:0/2703550571 wait complete. 2020-11-20T13:56:53.189 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 16 2020-11-20T13:56:53.398 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:33.416903+0000', 'modified': '2020-11-20T13:56:52.226497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5242': {'gid': 5242, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6835/3369439478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3369439478}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3369439478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:53.398 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:53.399 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:53.526 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":5242,"want_state":"up:active","state":"up:active","rank_uptime":4.2931451689999998,"mdsmap_epoch":16,"osdmap_epoch":37,"osdmap_epoch_barrier":37,"uptime":5.2085282299999998} 2020-11-20T13:56:53.526 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 5242, "mdsmap_epoch": 16, "osdmap_epoch": 37, "osdmap_epoch_barrier": 37, "rank_uptime": 4.293145169, "state": "up:active", "uptime": 5.20852823, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:53.527 INFO:teuthology.orchestra.run.smithi117:> rados -p cephfs_metadata rmxattr 10000000000.00000000 parent 2020-11-20T13:56:53.570 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.563+0000 7fdeaa09acc0 1 Processor -- start 2020-11-20T13:56:53.571 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.563+0000 7fdeaa09acc0 1 -- start start 2020-11-20T13:56:53.571 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634062a4f10 0x56340628d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x563406336d00 0x563406267220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634063370e0 0x563406267710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634061e6080 con 0x5634062a4f10 2020-11-20T13:56:53.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340628cee0 con 0x5634063370e0 2020-11-20T13:56:53.572 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x56340629edb0 con 0x563406336d00 2020-11-20T13:56:53.573 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634063370e0 0x563406267710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.573 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634063370e0 0x563406267710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42054/0 (socket says 172.21.15.117:42054) 2020-11-20T13:56:53.573 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 -- 172.21.15.117:0/2049974198 learned_addr learned my addr 172.21.15.117:0/2049974198 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:53.573 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 -- 172.21.15.117:0/2049974198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x563406336d00 msgr2=0x563406267220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:53.574 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 --2- 172.21.15.117:0/2049974198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x563406336d00 0x563406267220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:53.574 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 -- 172.21.15.117:0/2049974198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634062a4f10 msgr2=0x56340628d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:53.574 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 --2- 172.21.15.117:0/2049974198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634062a4f10 0x56340628d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:53.574 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 -- 172.21.15.117:0/2049974198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563406209380 con 0x5634063370e0 2020-11-20T13:56:53.574 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde9ba29700 1 --2- 172.21.15.117:0/2049974198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634063370e0 0x563406267710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fde94000f00 tx=0x7fde94006850).ready entity=mon.1 client_cookie=f177529fdbc98c12 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.575 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde99224700 1 -- 172.21.15.117:0/2049974198 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde940062e0 con 0x5634063370e0 2020-11-20T13:56:53.575 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde99224700 1 -- 172.21.15.117:0/2049974198 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde94004500 con 0x5634063370e0 2020-11-20T13:56:53.575 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fde99224700 1 -- 172.21.15.117:0/2049974198 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde94004860 con 0x5634063370e0 2020-11-20T13:56:53.575 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/2049974198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634063370e0 msgr2=0x563406267710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.575 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/2049974198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634063370e0 0x563406267710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fde94000f00 tx=0x7fde94006850).stop 2020-11-20T13:56:53.576 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/2049974198 shutdown_connections 2020-11-20T13:56:53.576 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/2049974198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5634063370e0 0x563406267710 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.576 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/2049974198 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634062a4f10 0x56340628d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:53.576 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/2049974198 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x563406336d00 0x563406267220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T13:56:53.577 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.564+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/2049974198 >> 172.21.15.117:0/2049974198 conn(0x5634062a9d70 msgr2=0x5634062a4350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.577 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.565+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/2049974198 shutdown_connections 2020-11-20T13:56:53.577 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.565+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/2049974198 wait complete. 2020-11-20T13:56:53.577 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.565+0000 7fdeaa09acc0 1 Processor -- start 2020-11-20T13:56:53.577 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.565+0000 7fdeaa09acc0 1 -- start start 2020-11-20T13:56:53.578 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x5634062a4f10 0x563406331e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.578 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563406336d00 0x56340628d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.579 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634063370e0 0x563406332790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.579 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340632c110 con 0x5634063370e0 2020-11-20T13:56:53.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563406332380 con 0x563406336d00 2020-11-20T13:56:53.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x56340634ab70 con 0x5634062a4f10 2020-11-20T13:56:53.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563406336d00 0x56340628d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563406336d00 0x56340628d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42058/0 (socket says 172.21.15.117:42058) 2020-11-20T13:56:53.580 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 -- 172.21.15.117:0/176217326 learned_addr learned my addr 172.21.15.117:0/176217326 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:53.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9b228700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x5634062a4f10 0x563406331e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9ba29700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634063370e0 0x563406332790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 -- 172.21.15.117:0/176217326 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x5634062a4f10 msgr2=0x563406331e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.581 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x5634062a4f10 0x563406331e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 -- 172.21.15.117:0/176217326 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634063370e0 msgr2=0x563406332790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x5634063370e0 0x563406332790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 -- 172.21.15.117:0/176217326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634061dee10 con 0x563406336d00 2020-11-20T13:56:53.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde9aa27700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563406336d00 0x56340628d2e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fde8c006000 tx=0x7fde8c005ae0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.582 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde98a23700 1 -- 172.21.15.117:0/176217326 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde8c00e070 con 0x563406336d00 2020-11-20T13:56:53.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde98a23700 1 -- 172.21.15.117:0/176217326 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c0077d0 con 0x563406336d00 2020-11-20T13:56:53.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56340634ae10 con 0x563406336d00 2020-11-20T13:56:53.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fde98a23700 1 -- 172.21.15.117:0/176217326 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde8c007b30 con 0x563406336d00 2020-11-20T13:56:53.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.566+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56340628d890 con 0x563406336d00 2020-11-20T13:56:53.583 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.567+0000 7fde98a23700 1 -- 172.21.15.117:0/176217326 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7fde8c019030 con 0x563406336d00 2020-11-20T13:56:53.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fde98a23700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fde7c04ff10 0x7fde7c0523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fde98a23700 1 -- 172.21.15.117:0/176217326 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fde8c067b40 con 0x563406336d00 2020-11-20T13:56:53.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fde9b228700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fde7c04ff10 0x7fde7c0523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x5634062694f0 0x5634063473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.584 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 --> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] -- osd_op(unknown.0.0:1 5.0 5:0cf6ad0f:::10000000000.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected e37) v8 -- 0x5634063478e0 con 0x5634062694f0 2020-11-20T13:56:53.585 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fde9b228700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fde7c04ff10 0x7fde7c0523d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fde84019000 tx=0x7fde84015000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fde9ba29700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x5634062694f0 0x5634063473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.586 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.568+0000 7fde9ba29700 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x5634062694f0 0x5634063473f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fde9ba29700 1 -- 172.21.15.117:0/176217326 <== osd.3 v2:172.21.15.117:6826/22922 1 ==== osd_op_reply(1 10000000000.00000000 [rmxattr] v37'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fde940062e0 con 0x5634062694f0 2020-11-20T13:56:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x5634062694f0 msgr2=0x5634063473f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6826/22922,v1:172.21.15.117:6827/22922] conn(0x5634062694f0 0x5634063473f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.596 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fde7c04ff10 msgr2=0x7fde7c0523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7fde7c04ff10 0x7fde7c0523d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fde84019000 tx=0x7fde84015000).stop 2020-11-20T13:56:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563406336d00 msgr2=0x56340628d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 --2- 172.21.15.117:0/176217326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563406336d00 0x56340628d2e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fde8c006000 tx=0x7fde8c005ae0).stop 2020-11-20T13:56:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fde9ba29700 1 -- 172.21.15.117:0/176217326 reap_dead start 2020-11-20T13:56:53.597 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 shutdown_connections 2020-11-20T13:56:53.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 >> 172.21.15.117:0/176217326 conn(0x5634062a9d70 msgr2=0x5634062a2780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 shutdown_connections 2020-11-20T13:56:53.598 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.570+0000 7fdeaa09acc0 1 -- 172.21.15.117:0/176217326 wait complete. 2020-11-20T13:56:53.599 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-11-20T13:56:53.742 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.740+0000 7f8177a20700 1 Processor -- start 2020-11-20T13:56:53.742 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.741+0000 7f8177a20700 1 -- start start 2020-11-20T13:56:53.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.741+0000 7f8177a20700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f81700ae8b0 0x7f81700aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.744 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.741+0000 7f8177a20700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f8170099fa0 0x7f817009a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.741+0000 7f8177a20700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81700b5120 con 0x7f81700b5860 2020-11-20T13:56:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.741+0000 7f8177a20700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81700b4f10 con 0x7f8170099fa0 2020-11-20T13:56:53.746 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.741+0000 7f8177a20700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81700b53c0 con 0x7f81700ae8b0 2020-11-20T13:56:53.752 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f81700ae8b0 0x7f81700aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f81700ae8b0 0x7f81700aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42066/0 (socket says 172.21.15.117:42066) 2020-11-20T13:56:53.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 -- 172.21.15.117:0/1130965544 learned_addr learned my addr 172.21.15.117:0/1130965544 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:53.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f8175fbd700 1 --2- 172.21.15.117:0/1130965544 >> v2:172.21.15.85:3300/0 conn(0x7f8170099fa0 0x7f817009a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 -- 172.21.15.117:0/1130965544 >> v2:172.21.15.85:3300/0 conn(0x7f8170099fa0 msgr2=0x7f817009a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:53.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 --2- 172.21.15.117:0/1130965544 >> v2:172.21.15.85:3300/0 conn(0x7f8170099fa0 0x7f817009a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 -- 172.21.15.117:0/1130965544 >> v1:172.21.15.85:6789/0 conn(0x7f81700b5860 legacy=0x7f8170098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:53.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 -- 172.21.15.117:0/1130965544 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700b4ac0 con 0x7f81700ae8b0 2020-11-20T13:56:53.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f81757bc700 1 --2- 172.21.15.117:0/1130965544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f81700aec90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f8170060ba0 tx=0x7f8160006e80).ready entity=mon.1 client_cookie=d10615e208f86039 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f815ffff700 1 -- 172.21.15.117:0/1130965544 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81600097b0 con 0x7f81700ae8b0 2020-11-20T13:56:53.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f815ffff700 1 -- 172.21.15.117:0/1130965544 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160009910 con 0x7f81700ae8b0 2020-11-20T13:56:53.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f815ffff700 1 -- 172.21.15.117:0/1130965544 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8160009a80 con 0x7f81700ae8b0 2020-11-20T13:56:53.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f8177a20700 1 -- 172.21.15.117:0/1130965544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 msgr2=0x7f81700aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.742+0000 7f8177a20700 1 --2- 172.21.15.117:0/1130965544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f81700aec90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f8170060ba0 tx=0x7f8160006e80).stop 2020-11-20T13:56:53.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.743+0000 7f8177a20700 1 -- 172.21.15.117:0/1130965544 shutdown_connections 2020-11-20T13:56:53.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.743+0000 7f8177a20700 1 --2- 172.21.15.117:0/1130965544 >> v2:172.21.15.85:3300/0 conn(0x7f8170099fa0 0x7f817009a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.743+0000 7f8177a20700 1 --2- 172.21.15.117:0/1130965544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f81700aec90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.743+0000 7f8177a20700 1 -- 172.21.15.117:0/1130965544 >> 172.21.15.117:0/1130965544 conn(0x7f817005b9f0 msgr2=0x7f817005be00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.743+0000 7f8177a20700 1 -- 172.21.15.117:0/1130965544 shutdown_connections 2020-11-20T13:56:53.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.743+0000 7f8177a20700 1 -- 172.21.15.117:0/1130965544 wait complete. 2020-11-20T13:56:53.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 Processor -- start 2020-11-20T13:56:53.761 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 -- start start 2020-11-20T13:56:53.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8170099fa0 0x7f81700b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f8170153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f81700b5860 0x7f8170157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.762 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170153ec0 con 0x7f81700b5860 2020-11-20T13:56:53.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170154160 con 0x7f81700ae8b0 2020-11-20T13:56:53.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8177a20700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8170154400 con 0x7f8170099fa0 2020-11-20T13:56:53.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8174fbb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f8170153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8174fbb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f8170153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42072/0 (socket says 172.21.15.117:42072) 2020-11-20T13:56:53.763 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.744+0000 7f8174fbb700 1 -- 172.21.15.117:0/3671225964 learned_addr learned my addr 172.21.15.117:0/3671225964 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f81757bc700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8170099fa0 0x7f81700b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8175fbd700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f81700b5860 0x7f8170157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8174fbb700 1 -- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8170099fa0 msgr2=0x7f81700b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8174fbb700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8170099fa0 0x7f81700b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.764 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8174fbb700 1 -- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f81700b5860 msgr2=0x7f8170157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8174fbb700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f81700b5860 0x7f8170157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8174fbb700 1 -- 172.21.15.117:0/3671225964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170064700 con 0x7f81700ae8b0 2020-11-20T13:56:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8174fbb700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f8170153580 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f816c001f50 tx=0x7f816c007c00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f815e7fc700 1 -- 172.21.15.117:0/3671225964 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816c00e4a0 con 0x7f81700ae8b0 2020-11-20T13:56:53.765 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f815e7fc700 1 -- 172.21.15.117:0/3671225964 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c00e600 con 0x7f81700ae8b0 2020-11-20T13:56:53.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8170153a70 con 0x7f81700ae8b0 2020-11-20T13:56:53.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f815e7fc700 1 -- 172.21.15.117:0/3671225964 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816c00e770 con 0x7f81700ae8b0 2020-11-20T13:56:53.766 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.745+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8170156e90 con 0x7f81700ae8b0 2020-11-20T13:56:53.767 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.746+0000 7f815e7fc700 1 -- 172.21.15.117:0/3671225964 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f816c010030 con 0x7f81700ae8b0 2020-11-20T13:56:53.780 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.746+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81701545c0 con 0x7f81700ae8b0 2020-11-20T13:56:53.781 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.746+0000 7f815e7fc700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8150051210 0x7f81500536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:53.781 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.746+0000 7f815e7fc700 1 -- 172.21.15.117:0/3671225964 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f816c068d40 con 0x7f81700ae8b0 2020-11-20T13:56:53.781 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.747+0000 7f81757bc700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8150051210 0x7f81500536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:53.782 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.747+0000 7f81757bc700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8150051210 0x7f81500536d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f816001b000 tx=0x7f8160016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:53.782 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.750+0000 7f815e7fc700 1 -- 172.21.15.117:0/3671225964 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f816c040f10 con 0x7f81700ae8b0 2020-11-20T13:56:53.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.898+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8170057050 con 0x7f81700ae8b0 2020-11-20T13:56:53.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.899+0000 7f815e7fc700 1 -- 172.21.15.117:0/3671225964 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16450 (secure 0 0 0) 0x7f816c044530 con 0x7f81700ae8b0 2020-11-20T13:56:53.900 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:53.901 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":37,"fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","created":"2020-11-20T13:55:30.198818+0000","modified":"2020-11-20T13:56:47.560030+0000","last_up_change":"2020-11-20T13:55:39.144730+0000","last_in_change":"2020-11-20T13:55:39.144730+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T13:55:34.161117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T13:55:42.995788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T13:56:31.475882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T13:56:32.365985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6425c8e9-c695-4b85-8225-09d3cf8763e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":22905},{"type":"v1","addr":"172.21.15.117:6803","nonce":22905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":22905},{"type":"v1","addr":"172.21.15.117:6805","nonce":22905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":22905},{"type":"v1","addr":"172.21.15.117:6809","nonce":22905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":22905},{"type":"v1","addr":"172.21.15.117:6807","nonce":22905}]},"public_addr":"172.21.15.117:6803/22905","cluster_addr":"172.21.15.117:6805/22905","heartbeat_back_addr":"172.21.15.117:6809/22905","heartbeat_front_addr":"172.21.15.117:6807/22905","state":["exists","up"]},{"osd":1,"uuid":"ac37f9fb-a4cd-4f26-9dc4-b4b7061b07bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":22907},{"type":"v1","addr":"172.21.15.117:6819","nonce":22907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":22907},{"type":"v1","addr":"172.21.15.117:6821","nonce":22907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":22907},{"type":"v1","addr":"172.21.15.117:6825","nonce":22907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":22907},{"type":"v1","addr":"172.21.15.117:6823","nonce":22907}]},"public_addr":"172.21.15.117:6819/22907","cluster_addr":"172.21.15.117:6821/22907","heartbeat_back_addr":"172.21.15.117:6825/22907","heartbeat_front_addr":"172.21.15.117:6823/22907","state":["exists","up"]},{"osd":2,"uuid":"2aaa1ba8-9363-4263-b86b-b163296df2dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":22906},{"type":"v1","addr":"172.21.15.117:6811","nonce":22906}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":22906},{"type":"v1","addr":"172.21.15.117:6813","nonce":22906}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":22906},{"type":"v1","addr":"172.21.15.117:6817","nonce":22906}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":22906},{"type":"v1","addr":"172.21.15.117:6815","nonce":22906}]},"public_addr":"172.21.15.117:6811/22906","cluster_addr":"172.21.15.117:6813/22906","heartbeat_back_addr":"172.21.15.117:6817/22906","heartbeat_front_addr":"172.21.15.117:6815/22906","state":["exists","up"]},{"osd":3,"uuid":"61687ded-23a5-4350-aa98-f5be6d396d11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":22922},{"type":"v1","addr":"172.21.15.117:6827","nonce":22922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":22922},{"type":"v1","addr":"172.21.15.117:6829","nonce":22922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":22922},{"type":"v1","addr":"172.21.15.117:6833","nonce":22922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":22922},{"type":"v1","addr":"172.21.15.117:6831","nonce":22922}]},"public_addr":"172.21.15.117:6827/22922","cluster_addr":"172.21.15.117:6829/22922","heartbeat_back_addr":"172.21.15.117:6833/22922","heartbeat_front_addr":"172.21.15.117:6831/22922","state":["exists","up"]},{"osd":4,"uuid":"48d54979-1a08-4759-846e-b56eb855d218","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":23126},{"type":"v1","addr":"172.21.15.85:6801","nonce":23126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":23126},{"type":"v1","addr":"172.21.15.85:6803","nonce":23126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":23126},{"type":"v1","addr":"172.21.15.85:6807","nonce":23126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":23126},{"type":"v1","addr":"172.21.15.85:6805","nonce":23126}]},"public_addr":"172.21.15.85:6801/23126","cluster_addr":"172.21.15.85:6803/23126","heartbeat_back_addr":"172.21.15.85:6807/23126","heartbeat_front_addr":"172.21.15.85:6805/23126","state":["exists","up"]},{"osd":5,"uuid":"8e801f28-c346-40d6-a3f8-5e4d072ed72d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":23124},{"type":"v1","addr":"172.21.15.85:6825","nonce":23124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":23124},{"type":"v1","addr":"172.21.15.85:6827","nonce":23124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":23124},{"type":"v1","addr":"172.21.15.85:6831","nonce":23124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":23124},{"type":"v1","addr":"172.21.15.85:6829","nonce":23124}]},"public_addr":"172.21.15.85:6825/23124","cluster_addr":"172.21.15.85:6827/23124","heartbeat_back_addr":"172.21.15.85:6831/23124","heartbeat_front_addr":"172.21.15.85:6829/23124","state":["exists","up"]},{"osd":6,"uuid":"94604f91-e895-43c8-8032-36a36e1191c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6815","nonce":23127},{"type":"v1","addr":"172.21.15.85:6817","nonce":23127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":23127},{"type":"v1","addr":"172.21.15.85:6819","nonce":23127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":23127},{"type":"v1","addr":"172.21.15.85:6823","nonce":23127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":23127},{"type":"v1","addr":"172.21.15.85:6821","nonce":23127}]},"public_addr":"172.21.15.85:6817/23127","cluster_addr":"172.21.15.85:6819/23127","heartbeat_back_addr":"172.21.15.85:6823/23127","heartbeat_front_addr":"172.21.15.85:6821/23127","state":["exists","up"]},{"osd":7,"uuid":"0b70a2e6-1111-426a-a7d9-5ccdd88b4038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":23128},{"type":"v1","addr":"172.21.15.85:6809","nonce":23128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":23128},{"type":"v1","addr":"172.21.15.85:6811","nonce":23128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":23128},{"type":"v1","addr":"172.21.15.85:6816","nonce":23128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":23128},{"type":"v1","addr":"172.21.15.85:6813","nonce":23128}]},"public_addr":"172.21.15.85:6809/23128","cluster_addr":"172.21.15.85:6811/23128","heartbeat_back_addr":"172.21.15.85:6816/23128","heartbeat_front_addr":"172.21.15.85:6813/23128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.219715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.210126+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.184385+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.237566+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.201769+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.247931+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.289808+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T13:55:37.198974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.117:6835/2328953218":"2020-11-21T13:56:47.551616+0000","172.21.15.117:6834/2328953218":"2020-11-21T13:56:47.551616+0000"},"erasure_code_profiles":{"cephfs_data":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-11-20T13:56:53.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8150051210 msgr2=0x7f81500536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8150051210 0x7f81500536d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f816001b000 tx=0x7f8160016000).stop 2020-11-20T13:56:53.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 msgr2=0x7f8170153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:53.903 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f8170153580 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f816c001f50 tx=0x7f816c007c00).stop 2020-11-20T13:56:53.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 shutdown_connections 2020-11-20T13:56:53.904 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f8150051210 0x7f81500536d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8170099fa0 0x7f81700b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f81700b5860 0x7f8170157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 --2- 172.21.15.117:0/3671225964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81700ae8b0 0x7f8170153580 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:53.905 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 >> 172.21.15.117:0/3671225964 conn(0x7f817005b9f0 msgr2=0x7f81700b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:53.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 shutdown_connections 2020-11-20T13:56:53.906 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:53.900+0000 7f8177a20700 1 -- 172.21.15.117:0/3671225964 wait complete. 2020-11-20T13:56:53.912 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T13:55:34.161117+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T13:55:42.995788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T13:56:31.475882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T13:56:32.365985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T13:56:53.914 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-11-20T13:56:54.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.023+0000 7f463ffff700 1 Processor -- start 2020-11-20T13:56:54.024 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.023+0000 7f463ffff700 1 -- start start 2020-11-20T13:56:54.026 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.023+0000 7f463ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f464009cdd0 0x7f4640147c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.023+0000 7f463ffff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f464005ee90 0x7f4640067200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463ffff700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4640066890 con 0x7f4640146840 2020-11-20T13:56:54.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463ffff700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4640148140 con 0x7f464005ee90 2020-11-20T13:56:54.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4640066ac0 con 0x7f464009cdd0 2020-11-20T13:56:54.027 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f464009cdd0 0x7f4640147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f464009cdd0 0x7f4640147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42078/0 (socket says 172.21.15.117:42078) 2020-11-20T13:56:54.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 -- 172.21.15.117:0/224631168 learned_addr learned my addr 172.21.15.117:0/224631168 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:54.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463e7fc700 1 --2- 172.21.15.117:0/224631168 >> v2:172.21.15.85:3300/0 conn(0x7f464005ee90 0x7f4640067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 -- 172.21.15.117:0/224631168 >> v2:172.21.15.85:3300/0 conn(0x7f464005ee90 msgr2=0x7f4640067200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.028 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 --2- 172.21.15.117:0/224631168 >> v2:172.21.15.85:3300/0 conn(0x7f464005ee90 0x7f4640067200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 -- 172.21.15.117:0/224631168 >> v1:172.21.15.85:6789/0 conn(0x7f4640146840 legacy=0x7f464009e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.029 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 -- 172.21.15.117:0/224631168 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640066d60 con 0x7f464009cdd0 2020-11-20T13:56:54.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463e7fc700 1 --2- 172.21.15.117:0/224631168 >> v2:172.21.15.85:3300/0 conn(0x7f464005ee90 0x7f4640067200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:54.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463effd700 1 --2- 172.21.15.117:0/224631168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f4640147c50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f46380032b0 tx=0x7f4638009270).ready entity=mon.1 client_cookie=d9569e25a5fa9869 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463dffb700 1 -- 172.21.15.117:0/224631168 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f463800bc00 con 0x7f464009cdd0 2020-11-20T13:56:54.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463dffb700 1 -- 172.21.15.117:0/224631168 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463800bd60 con 0x7f464009cdd0 2020-11-20T13:56:54.030 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463dffb700 1 -- 172.21.15.117:0/224631168 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4638011400 con 0x7f464009cdd0 2020-11-20T13:56:54.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463ffff700 1 -- 172.21.15.117:0/224631168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 msgr2=0x7f4640147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.024+0000 7f463ffff700 1 --2- 172.21.15.117:0/224631168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f4640147c50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f46380032b0 tx=0x7f4638009270).stop 2020-11-20T13:56:54.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.025+0000 7f463ffff700 1 -- 172.21.15.117:0/224631168 shutdown_connections 2020-11-20T13:56:54.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.025+0000 7f463ffff700 1 --2- 172.21.15.117:0/224631168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f4640147c50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.031 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.025+0000 7f463ffff700 1 --2- 172.21.15.117:0/224631168 >> v2:172.21.15.85:3300/0 conn(0x7f464005ee90 0x7f4640067200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.025+0000 7f463ffff700 1 -- 172.21.15.117:0/224631168 >> 172.21.15.117:0/224631168 conn(0x7f46400bd890 msgr2=0x7f46400bdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.025+0000 7f463ffff700 1 -- 172.21.15.117:0/224631168 shutdown_connections 2020-11-20T13:56:54.032 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.025+0000 7f463ffff700 1 -- 172.21.15.117:0/224631168 wait complete. 2020-11-20T13:56:54.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.026+0000 7f463ffff700 1 Processor -- start 2020-11-20T13:56:54.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.026+0000 7f463ffff700 1 -- start start 2020-11-20T13:56:54.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.026+0000 7f463ffff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f464005ee90 0x7f464014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.026+0000 7f463ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f46400b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463ffff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4640146840 0x7f46400b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463ffff700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464013c830 con 0x7f4640146840 2020-11-20T13:56:54.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46400b3fd0 con 0x7f464009cdd0 2020-11-20T13:56:54.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463ffff700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46400b2eb0 con 0x7f464005ee90 2020-11-20T13:56:54.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f46400b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f46400b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42084/0 (socket says 172.21.15.117:42084) 2020-11-20T13:56:54.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 -- 172.21.15.117:0/1321616085 learned_addr learned my addr 172.21.15.117:0/1321616085 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:54.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463effd700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f464005ee90 0x7f464014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463f7fe700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4640146840 0x7f46400b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 -- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f464005ee90 msgr2=0x7f464014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f464005ee90 0x7f464014aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.040 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 -- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4640146840 msgr2=0x7f46400b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.041 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4640146840 0x7f46400b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 -- 172.21.15.117:0/1321616085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400b3150 con 0x7f464009cdd0 2020-11-20T13:56:54.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463f7fe700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4640146840 0x7f46400b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:54.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463e7fc700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f46400b35f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f462800a000 tx=0x7f4628008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.048 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f4637fff700 1 -- 172.21.15.117:0/1321616085 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4628009980 con 0x7f464009cdd0 2020-11-20T13:56:54.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f4637fff700 1 -- 172.21.15.117:0/1321616085 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4628009ae0 con 0x7f464009cdd0 2020-11-20T13:56:54.049 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f4637fff700 1 -- 172.21.15.117:0/1321616085 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4628009e40 con 0x7f464009cdd0 2020-11-20T13:56:54.050 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464014d550 con 0x7f464009cdd0 2020-11-20T13:56:54.050 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.027+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464015a670 con 0x7f464009cdd0 2020-11-20T13:56:54.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.028+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464015a830 con 0x7f464009cdd0 2020-11-20T13:56:54.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.028+0000 7f4637fff700 1 -- 172.21.15.117:0/1321616085 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f4628013480 con 0x7f464009cdd0 2020-11-20T13:56:54.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.029+0000 7f4637fff700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4624051210 0x7f46240536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.029+0000 7f4637fff700 1 -- 172.21.15.117:0/1321616085 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f4628069950 con 0x7f464009cdd0 2020-11-20T13:56:54.051 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.029+0000 7f463effd700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4624051210 0x7f46240536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.029+0000 7f463effd700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4624051210 0x7f46240536d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4638018000 tx=0x7f4638016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.052 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.032+0000 7f4637fff700 1 -- 172.21.15.117:0/1321616085 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f464015a830 con 0x7f464009cdd0 2020-11-20T13:56:54.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.184+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f464015a800 con 0x7f464009cdd0 2020-11-20T13:56:54.186 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.184+0000 7f4637fff700 1 -- 172.21.15.117:0/1321616085 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v37) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4628041ac0 con 0x7f464009cdd0 2020-11-20T13:56:54.186 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2020-11-20T13:56:54.188 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4624051210 msgr2=0x7f46240536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4624051210 0x7f46240536d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f4638018000 tx=0x7f4638016000).stop 2020-11-20T13:56:54.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 msgr2=0x7f46400b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.189 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f46400b35f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f462800a000 tx=0x7f4628008040).stop 2020-11-20T13:56:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 shutdown_connections 2020-11-20T13:56:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f4624051210 0x7f46240536d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f464005ee90 0x7f464014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.190 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4640146840 0x7f46400b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 --2- 172.21.15.117:0/1321616085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f464009cdd0 0x7f46400b35f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 >> 172.21.15.117:0/1321616085 conn(0x7f46400bd890 msgr2=0x7f4640148290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 shutdown_connections 2020-11-20T13:56:54.191 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.186+0000 7f463ffff700 1 -- 172.21.15.117:0/1321616085 wait complete. 2020-11-20T13:56:54.198 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-11-20T13:56:54.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.306+0000 7f9037ff6700 1 Processor -- start 2020-11-20T13:56:54.308 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.306+0000 7f9037ff6700 1 -- start start 2020-11-20T13:56:54.310 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9037ff6700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9030098d60 0x7f903009b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9037ff6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300b5240 0x7f90300630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9037ff6700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90300b4860 con 0x7f90300635b0 2020-11-20T13:56:54.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9037ff6700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90301439f0 con 0x7f9030098d60 2020-11-20T13:56:54.311 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9037ff6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9030062f40 con 0x7f90300b5240 2020-11-20T13:56:54.314 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9035d92700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9030098d60 0x7f903009b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9035d92700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9030098d60 0x7f903009b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42090/0 (socket says 172.21.15.117:42090) 2020-11-20T13:56:54.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.307+0000 7f9035d92700 1 -- 172.21.15.117:0/527488264 learned_addr learned my addr 172.21.15.117:0/527488264 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:54.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9035591700 1 --2- 172.21.15.117:0/527488264 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300b5240 0x7f90300630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9035d92700 1 -- 172.21.15.117:0/527488264 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300b5240 msgr2=0x7f90300630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.315 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9035d92700 1 --2- 172.21.15.117:0/527488264 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300b5240 0x7f90300630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9035d92700 1 -- 172.21.15.117:0/527488264 >> v1:172.21.15.85:6789/0 conn(0x7f90300635b0 legacy=0x7f9030062830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9035d92700 1 -- 172.21.15.117:0/527488264 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300b4b00 con 0x7f9030098d60 2020-11-20T13:56:54.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9035d92700 1 --2- 172.21.15.117:0/527488264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030098d60 0x7f903009b130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f9030060c50 tx=0x7f9020006e80).ready entity=mon.1 client_cookie=64fdefa22308daaa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9034d90700 1 -- 172.21.15.117:0/527488264 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9020009770 con 0x7f9030098d60 2020-11-20T13:56:54.316 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9034d90700 1 -- 172.21.15.117:0/527488264 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90200098d0 con 0x7f9030098d60 2020-11-20T13:56:54.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9034d90700 1 -- 172.21.15.117:0/527488264 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9020009a40 con 0x7f9030098d60 2020-11-20T13:56:54.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 -- 172.21.15.117:0/527488264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030098d60 msgr2=0x7f903009b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 --2- 172.21.15.117:0/527488264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030098d60 0x7f903009b130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f9030060c50 tx=0x7f9020006e80).stop 2020-11-20T13:56:54.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 -- 172.21.15.117:0/527488264 shutdown_connections 2020-11-20T13:56:54.317 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 --2- 172.21.15.117:0/527488264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030098d60 0x7f903009b130 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 --2- 172.21.15.117:0/527488264 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300b5240 0x7f90300630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 -- 172.21.15.117:0/527488264 >> 172.21.15.117:0/527488264 conn(0x7f90300b9770 msgr2=0x7f90300b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 -- 172.21.15.117:0/527488264 shutdown_connections 2020-11-20T13:56:54.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.308+0000 7f9037ff6700 1 -- 172.21.15.117:0/527488264 wait complete. 2020-11-20T13:56:54.318 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.309+0000 7f9037ff6700 1 Processor -- start 2020-11-20T13:56:54.319 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.309+0000 7f9037ff6700 1 -- start start 2020-11-20T13:56:54.323 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9037ff6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300635b0 0x7f9030090dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9037ff6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9030098d60 0x7f9030067380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9037ff6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300b5240 0x7f9030157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.324 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9037ff6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90300903f0 con 0x7f9030098d60 2020-11-20T13:56:54.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9037ff6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9030090690 con 0x7f90300b5240 2020-11-20T13:56:54.325 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9037ff6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9030090930 con 0x7f90300635b0 2020-11-20T13:56:54.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300b5240 0x7f9030157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300b5240 0x7f9030157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42100/0 (socket says 172.21.15.117:42100) 2020-11-20T13:56:54.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 -- 172.21.15.117:0/4119331964 learned_addr learned my addr 172.21.15.117:0/4119331964 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:54.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9035591700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9030098d60 0x7f9030067380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.326 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9035d92700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300635b0 0x7f9030090dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 -- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300635b0 msgr2=0x7f9030090dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300635b0 0x7f9030090dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 -- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9030098d60 msgr2=0x7f9030067380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9030098d60 0x7f9030067380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.327 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.310+0000 7f9036593700 1 -- 172.21.15.117:0/4119331964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9030090070 con 0x7f90300b5240 2020-11-20T13:56:54.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.311+0000 7f9036593700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300b5240 0x7f9030157ec0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f90280022d0 tx=0x7f9028007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.311+0000 7f901effd700 1 -- 172.21.15.117:0/4119331964 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f902800a5a0 con 0x7f90300b5240 2020-11-20T13:56:54.328 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.311+0000 7f901effd700 1 -- 172.21.15.117:0/4119331964 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902800a700 con 0x7f90300b5240 2020-11-20T13:56:54.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.311+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90301583b0 con 0x7f90300b5240 2020-11-20T13:56:54.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.311+0000 7f901effd700 1 -- 172.21.15.117:0/4119331964 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f902800aa60 con 0x7f90300b5240 2020-11-20T13:56:54.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.311+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9030157970 con 0x7f90300b5240 2020-11-20T13:56:54.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.312+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90301586e0 con 0x7f90300b5240 2020-11-20T13:56:54.329 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.312+0000 7f901effd700 1 -- 172.21.15.117:0/4119331964 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f9028019030 con 0x7f90300b5240 2020-11-20T13:56:54.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.312+0000 7f901effd700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9010051290 0x7f9010053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.312+0000 7f901effd700 1 -- 172.21.15.117:0/4119331964 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9028068ae0 con 0x7f90300b5240 2020-11-20T13:56:54.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.312+0000 7f9035d92700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9010051290 0x7f9010053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.313+0000 7f9035d92700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9010051290 0x7f9010053750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f902001b000 tx=0x7f9020016000).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.330 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.316+0000 7f901effd700 1 -- 172.21.15.117:0/4119331964 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f9028040ba0 con 0x7f90300b5240 2020-11-20T13:56:54.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.465+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9030057050 con 0x7f90300b5240 2020-11-20T13:56:54.467 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.466+0000 7f901effd700 1 -- 172.21.15.117:0/4119331964 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90280441c0 con 0x7f90300b5240 2020-11-20T13:56:54.467 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:54.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9010051290 msgr2=0x7f9010053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9010051290 0x7f9010053750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f902001b000 tx=0x7f9020016000).stop 2020-11-20T13:56:54.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300b5240 msgr2=0x7f9030157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.469 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300b5240 0x7f9030157ec0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f90280022d0 tx=0x7f9028007ef0).stop 2020-11-20T13:56:54.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 shutdown_connections 2020-11-20T13:56:54.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f9010051290 0x7f9010053750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f90300635b0 0x7f9030090dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.470 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300b5240 0x7f9030157ec0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 --2- 172.21.15.117:0/4119331964 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9030098d60 0x7f9030067380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 >> 172.21.15.117:0/4119331964 conn(0x7f90300b9770 msgr2=0x7f9030099440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 shutdown_connections 2020-11-20T13:56:54.471 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.467+0000 7f9037ff6700 1 -- 172.21.15.117:0/4119331964 wait complete. 2020-11-20T13:56:54.486 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-11-20T13:56:54.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.593+0000 7f7d9e630700 1 Processor -- start 2020-11-20T13:56:54.595 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.593+0000 7f7d9e630700 1 -- start start 2020-11-20T13:56:54.600 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9e630700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7d9805f300 0x7f7d98062b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9e630700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f7d9809bf70 0x7f7d98146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.601 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9e630700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98146e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9e630700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9809ee60 con 0x7f7d9809bf70 2020-11-20T13:56:54.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9e630700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9809f100 con 0x7f7d9805f300 2020-11-20T13:56:54.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9e630700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9809f3a0 con 0x7f7d98062430 2020-11-20T13:56:54.602 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9d62e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7d9805f300 0x7f7d98062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9d62e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7d9805f300 0x7f7d98062b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42104/0 (socket says 172.21.15.117:42104) 2020-11-20T13:56:54.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1635614250 learned_addr learned my addr 172.21.15.117:0/1635614250 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:54.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9de2f700 1 --2- 172.21.15.117:0/1635614250 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9ce2d700 1 --2- 172.21.15.117:0/1635614250 >> v2:172.21.15.85:3300/0 conn(0x7f7d9809bf70 0x7f7d98146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.603 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.594+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1635614250 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 msgr2=0x7f7d98146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9d62e700 1 --2- 172.21.15.117:0/1635614250 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98146e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1635614250 >> v2:172.21.15.85:3300/0 conn(0x7f7d9809bf70 msgr2=0x7f7d98146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9d62e700 1 --2- 172.21.15.117:0/1635614250 >> v2:172.21.15.85:3300/0 conn(0x7f7d9809bf70 0x7f7d98146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1635614250 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9805f6e0 con 0x7f7d9805f300 2020-11-20T13:56:54.604 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9de2f700 1 --2- 172.21.15.117:0/1635614250 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:54.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9d62e700 1 --2- 172.21.15.117:0/1635614250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d98062b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f7d88002f30 tx=0x7f7d88006e80).ready entity=mon.1 client_cookie=e47bf1cbf8930579 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d87fff700 1 -- 172.21.15.117:0/1635614250 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d880097f0 con 0x7f7d9805f300 2020-11-20T13:56:54.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d87fff700 1 -- 172.21.15.117:0/1635614250 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d88009950 con 0x7f7d9805f300 2020-11-20T13:56:54.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d87fff700 1 -- 172.21.15.117:0/1635614250 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d88009cb0 con 0x7f7d9805f300 2020-11-20T13:56:54.605 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 -- 172.21.15.117:0/1635614250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 msgr2=0x7f7d98062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1635614250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d98062b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f7d88002f30 tx=0x7f7d88006e80).stop 2020-11-20T13:56:54.606 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 -- 172.21.15.117:0/1635614250 shutdown_connections 2020-11-20T13:56:54.609 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1635614250 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1635614250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d98062b50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1635614250 >> v2:172.21.15.85:3300/0 conn(0x7f7d9809bf70 0x7f7d98146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 -- 172.21.15.117:0/1635614250 >> 172.21.15.117:0/1635614250 conn(0x7f7d980b9490 msgr2=0x7f7d980b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.610 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 -- 172.21.15.117:0/1635614250 shutdown_connections 2020-11-20T13:56:54.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.595+0000 7f7d9e630700 1 -- 172.21.15.117:0/1635614250 wait complete. 2020-11-20T13:56:54.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.596+0000 7f7d9e630700 1 Processor -- start 2020-11-20T13:56:54.611 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.596+0000 7f7d9e630700 1 -- start start 2020-11-20T13:56:54.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9e630700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d9813fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.615 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9e630700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9e630700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7d9809bf70 0x7f7d98159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9e630700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d981343f0 con 0x7f7d9809bf70 2020-11-20T13:56:54.617 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9e630700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9813f0a0 con 0x7f7d9805f300 2020-11-20T13:56:54.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9e630700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9813f340 con 0x7f7d98062430 2020-11-20T13:56:54.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d9813fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.618 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d9813fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42110/0 (socket says 172.21.15.117:42110) 2020-11-20T13:56:54.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1674099055 learned_addr learned my addr 172.21.15.117:0/1674099055 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:54.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9ce2d700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.619 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9de2f700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7d9809bf70 0x7f7d98159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 msgr2=0x7f7d98145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.620 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7d9809bf70 msgr2=0x7f7d98159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7d9809bf70 0x7f7d98159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 -- 172.21.15.117:0/1674099055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9813cfb0 con 0x7f7d9805f300 2020-11-20T13:56:54.627 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9de2f700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7d9809bf70 0x7f7d98159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:54.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.597+0000 7f7d9d62e700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d9813fa10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f7d88002f30 tx=0x7f7d88015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.598+0000 7f7d867fc700 1 -- 172.21.15.117:0/1674099055 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d88011c30 con 0x7f7d9805f300 2020-11-20T13:56:54.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.598+0000 7f7d867fc700 1 -- 172.21.15.117:0/1674099055 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d88011d90 con 0x7f7d9805f300 2020-11-20T13:56:54.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.598+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9813ed20 con 0x7f7d9805f300 2020-11-20T13:56:54.628 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.598+0000 7f7d867fc700 1 -- 172.21.15.117:0/1674099055 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d8801e650 con 0x7f7d9805f300 2020-11-20T13:56:54.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.598+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d98159920 con 0x7f7d9805f300 2020-11-20T13:56:54.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.599+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d98057050 con 0x7f7d9805f300 2020-11-20T13:56:54.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.599+0000 7f7d867fc700 1 -- 172.21.15.117:0/1674099055 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f7d88025030 con 0x7f7d9805f300 2020-11-20T13:56:54.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.599+0000 7f7d867fc700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7d78051290 0x7f7d78053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.629 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.599+0000 7f7d867fc700 1 -- 172.21.15.117:0/1674099055 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7d8806a420 con 0x7f7d9805f300 2020-11-20T13:56:54.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.599+0000 7f7d9ce2d700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7d78051290 0x7f7d78053750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.599+0000 7f7d9ce2d700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7d78051290 0x7f7d78053750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f7d9400c000 tx=0x7f7d94008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.630 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.602+0000 7f7d867fc700 1 -- 172.21.15.117:0/1674099055 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f7d88042670 con 0x7f7d9805f300 2020-11-20T13:56:54.753 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.752+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d9805ca00 con 0x7f7d9805f300 2020-11-20T13:56:54.754 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.752+0000 7f7d867fc700 1 -- 172.21.15.117:0/1674099055 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7d880230a0 con 0x7f7d9805f300 2020-11-20T13:56:54.754 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:54.755 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7d78051290 msgr2=0x7f7d78053750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7d78051290 0x7f7d78053750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f7d9400c000 tx=0x7f7d94008040).stop 2020-11-20T13:56:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 msgr2=0x7f7d9813fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d9813fa10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f7d88002f30 tx=0x7f7d88015000).stop 2020-11-20T13:56:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 shutdown_connections 2020-11-20T13:56:54.756 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7d78051290 0x7f7d78053750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f7d9809bf70 0x7f7d98159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d9805f300 0x7f7d9813fa10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 --2- 172.21.15.117:0/1674099055 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7d98062430 0x7f7d98145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.757 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 >> 172.21.15.117:0/1674099055 conn(0x7f7d980b9490 msgr2=0x7f7d980b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 shutdown_connections 2020-11-20T13:56:54.758 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.754+0000 7f7d9e630700 1 -- 172.21.15.117:0/1674099055 wait complete. 2020-11-20T13:56:54.765 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-11-20T13:56:54.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.873+0000 7f714fa88700 1 Processor -- start 2020-11-20T13:56:54.874 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.873+0000 7f714fa88700 1 -- start start 2020-11-20T13:56:54.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714fa88700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7148143eb0 0x7f71480b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714fa88700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480999d0 con 0x7f714809bd80 2020-11-20T13:56:54.876 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714fa88700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7148099c70 con 0x7f714809cb40 2020-11-20T13:56:54.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714fa88700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71480997f0 con 0x7f7148143eb0 2020-11-20T13:56:54.877 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714e025700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7148143eb0 0x7f71480b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714d023700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f714809cb40 0x7f71481437a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:33048/0 (socket says 172.21.15.117:33048) 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714d023700 1 -- 172.21.15.117:0/2293654246 learned_addr learned my addr 172.21.15.117:0/2293654246 (peer_addr_for_me v1:172.21.15.117:0/0) 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714e025700 1 -- 172.21.15.117:0/2293654246 >> v1:172.21.15.117:6789/0 conn(0x7f714809cb40 legacy=0x7f71481437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.878 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714e025700 1 -- 172.21.15.117:0/2293654246 >> v1:172.21.15.85:6789/0 conn(0x7f714809bd80 legacy=0x7f714809a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714c822700 1 -- 172.21.15.117:0/2293654246 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786333423 0 0) 0x7f7148099c70 con 0x7f714809cb40 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714e025700 1 -- 172.21.15.117:0/2293654246 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714809c160 con 0x7f7148143eb0 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714e025700 1 --2- 172.21.15.117:0/2293654246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7148143eb0 0x7f71480b4110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f714809ad40 tx=0x7f71400091a0).ready entity=mon.1 client_cookie=1cdd4e0f96eb1fe3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714c822700 1 -- 172.21.15.117:0/2293654246 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714000ba10 con 0x7f7148143eb0 2020-11-20T13:56:54.879 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714c822700 1 -- 172.21.15.117:0/2293654246 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714000bb70 con 0x7f7148143eb0 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714c822700 1 -- 172.21.15.117:0/2293654246 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714000bce0 con 0x7f7148143eb0 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714fa88700 1 -- 172.21.15.117:0/2293654246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7148143eb0 msgr2=0x7f71480b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.874+0000 7f714fa88700 1 --2- 172.21.15.117:0/2293654246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7148143eb0 0x7f71480b4110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f714809ad40 tx=0x7f71400091a0).stop 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.875+0000 7f714fa88700 1 -- 172.21.15.117:0/2293654246 shutdown_connections 2020-11-20T13:56:54.880 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.875+0000 7f714fa88700 1 --2- 172.21.15.117:0/2293654246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7148143eb0 0x7f71480b4110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.875+0000 7f714fa88700 1 -- 172.21.15.117:0/2293654246 >> 172.21.15.117:0/2293654246 conn(0x7f71480b9770 msgr2=0x7f71480b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:54.885 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.875+0000 7f714fa88700 1 -- 172.21.15.117:0/2293654246 shutdown_connections 2020-11-20T13:56:54.886 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.875+0000 7f714fa88700 1 -- 172.21.15.117:0/2293654246 wait complete. 2020-11-20T13:56:54.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 Processor -- start 2020-11-20T13:56:54.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 -- start start 2020-11-20T13:56:54.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714809bd80 0x7f7148138e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.890 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f714809cb40 0x7f714809e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7148143eb0 0x7f7148137290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b2ae0 con 0x7f714809cb40 2020-11-20T13:56:54.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b2d80 con 0x7f714809bd80 2020-11-20T13:56:54.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714fa88700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480b3020 con 0x7f7148143eb0 2020-11-20T13:56:54.891 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.876+0000 7f714d824700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714809bd80 0x7f7148138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714809bd80 0x7f7148138e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42126/0 (socket says 172.21.15.117:42126) 2020-11-20T13:56:54.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 -- 172.21.15.117:0/2140574729 learned_addr learned my addr 172.21.15.117:0/2140574729 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:54.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714e025700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7148143eb0 0x7f7148137290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.892 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d023700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f714809cb40 0x7f714809e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.893 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 -- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7148143eb0 msgr2=0x7f7148137290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7148143eb0 0x7f7148137290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 -- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f714809cb40 msgr2=0x7f714809e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f714809cb40 0x7f714809e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:54.899 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 -- 172.21.15.117:0/2140574729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480b32c0 con 0x7f714809bd80 2020-11-20T13:56:54.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714e025700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7148143eb0 0x7f7148137290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:54.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d023700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f714809cb40 0x7f714809e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:54.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714d824700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714809bd80 0x7f7148138e30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f7138001f50 tx=0x7f7138007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f7136ffd700 1 -- 172.21.15.117:0/2140574729 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713800e4a0 con 0x7f714809bd80 2020-11-20T13:56:54.900 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f7136ffd700 1 -- 172.21.15.117:0/2140574729 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713800e600 con 0x7f714809bd80 2020-11-20T13:56:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714809eec0 con 0x7f714809bd80 2020-11-20T13:56:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f7136ffd700 1 -- 172.21.15.117:0/2140574729 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f713800e960 con 0x7f714809bd80 2020-11-20T13:56:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.877+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714813c8b0 con 0x7f714809bd80 2020-11-20T13:56:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.878+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7148137b00 con 0x7f714809bd80 2020-11-20T13:56:54.901 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.878+0000 7f7136ffd700 1 -- 172.21.15.117:0/2140574729 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f7138019030 con 0x7f714809bd80 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.879+0000 7f7136ffd700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7128051210 0x7f71280536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.879+0000 7f7136ffd700 1 -- 172.21.15.117:0/2140574729 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f7138068610 con 0x7f714809bd80 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.879+0000 7f714d023700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7128051210 0x7f71280536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.879+0000 7f714d023700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7128051210 0x7f71280536d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f7144008000 tx=0x7f7144009040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:54.902 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:54.882+0000 7f7136ffd700 1 -- 172.21.15.117:0/2140574729 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f71380407f0 con 0x7f714809bd80 2020-11-20T13:56:55.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.032+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7148057050 con 0x7f714809bd80 2020-11-20T13:56:55.034 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.033+0000 7f7136ffd700 1 -- 172.21.15.117:0/2140574729 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7138043e10 con 0x7f714809bd80 2020-11-20T13:56:55.034 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2020-11-20T13:56:55.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7128051210 msgr2=0x7f71280536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7128051210 0x7f71280536d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f7144008000 tx=0x7f7144009040).stop 2020-11-20T13:56:55.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714809bd80 msgr2=0x7f7148138e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.036 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714809bd80 0x7f7148138e30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f7138001f50 tx=0x7f7138007b70).stop 2020-11-20T13:56:55.037 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 shutdown_connections 2020-11-20T13:56:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f7128051210 0x7f71280536d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f7148143eb0 0x7f7148137290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714809bd80 0x7f7148138e30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 --2- 172.21.15.117:0/2140574729 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f714809cb40 0x7f714809e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.038 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 >> 172.21.15.117:0/2140574729 conn(0x7f71480b9770 msgr2=0x7f714813b660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 shutdown_connections 2020-11-20T13:56:55.039 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.034+0000 7f714fa88700 1 -- 172.21.15.117:0/2140574729 wait complete. 2020-11-20T13:56:55.049 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T13:56:55.049 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-11-20T13:56:55.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.154+0000 7f332ebcd700 1 Processor -- start 2020-11-20T13:56:55.156 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.154+0000 7f332ebcd700 1 -- start start 2020-11-20T13:56:55.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f332ebcd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332805cd80 0x7f3328147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f332ebcd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f332814a830 0x7f3328140d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.158 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f332ebcd700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3328141710 con 0x7f3328141260 2020-11-20T13:56:55.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f332ebcd700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33280b6780 con 0x7f332814a830 2020-11-20T13:56:55.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f332ebcd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3328147850 con 0x7f332805cd80 2020-11-20T13:56:55.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f3327fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f332814a830 0x7f3328140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f3327fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f332814a830 0x7f3328140d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42134/0 (socket says 172.21.15.117:42134) 2020-11-20T13:56:55.159 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f3327fff700 1 -- 172.21.15.117:0/3648010531 learned_addr learned my addr 172.21.15.117:0/3648010531 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:55.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f332d16a700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3328141260 0x7f3328060630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:33064/0 (socket says 172.21.15.117:33064) 2020-11-20T13:56:55.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f332c969700 1 --2- 172.21.15.117:0/3648010531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332805cd80 0x7f3328147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f3327fff700 1 -- 172.21.15.117:0/3648010531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332805cd80 msgr2=0x7f3328147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T13:56:55.160 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f3327fff700 1 --2- 172.21.15.117:0/3648010531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332805cd80 0x7f3328147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f3327fff700 1 -- 172.21.15.117:0/3648010531 >> v1:172.21.15.117:6789/0 conn(0x7f3328141260 legacy=0x7f3328060630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.155+0000 7f3327fff700 1 -- 172.21.15.117:0/3648010531 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328147af0 con 0x7f332814a830 2020-11-20T13:56:55.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332c969700 1 --2- 172.21.15.117:0/3648010531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332805cd80 0x7f3328147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T13:56:55.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f3327fff700 1 --2- 172.21.15.117:0/3648010531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332814a830 0x7f3328140d70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f3314002f30 tx=0x7f3314006e80).ready entity=mon.1 client_cookie=da1060c26a42efe4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.161 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f33277fe700 1 -- 172.21.15.117:0/3648010531 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33140097f0 con 0x7f332814a830 2020-11-20T13:56:55.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f33277fe700 1 -- 172.21.15.117:0/3648010531 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3314009950 con 0x7f332814a830 2020-11-20T13:56:55.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f33277fe700 1 -- 172.21.15.117:0/3648010531 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3314009cb0 con 0x7f332814a830 2020-11-20T13:56:55.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 -- 172.21.15.117:0/3648010531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332814a830 msgr2=0x7f3328140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.162 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 --2- 172.21.15.117:0/3648010531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332814a830 0x7f3328140d70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f3314002f30 tx=0x7f3314006e80).stop 2020-11-20T13:56:55.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 -- 172.21.15.117:0/3648010531 shutdown_connections 2020-11-20T13:56:55.163 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 --2- 172.21.15.117:0/3648010531 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332805cd80 0x7f3328147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 --2- 172.21.15.117:0/3648010531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f332814a830 0x7f3328140d70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 -- 172.21.15.117:0/3648010531 >> 172.21.15.117:0/3648010531 conn(0x7f33280b7480 msgr2=0x7f3328062f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 -- 172.21.15.117:0/3648010531 shutdown_connections 2020-11-20T13:56:55.169 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.156+0000 7f332ebcd700 1 -- 172.21.15.117:0/3648010531 wait complete. 2020-11-20T13:56:55.172 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.157+0000 7f332ebcd700 1 Processor -- start 2020-11-20T13:56:55.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.157+0000 7f332ebcd700 1 -- start start 2020-11-20T13:56:55.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332ebcd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f332805cd80 0x7f332815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.173 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332ebcd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328141260 0x7f332813fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332ebcd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332814a830 0x7f3328156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.174 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332ebcd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3328134c40 con 0x7f332805cd80 2020-11-20T13:56:55.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332ebcd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332809d1a0 con 0x7f3328141260 2020-11-20T13:56:55.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332ebcd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332809d440 con 0x7f332814a830 2020-11-20T13:56:55.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f3327fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328141260 0x7f332813fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.178 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f3327fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328141260 0x7f332813fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42140/0 (socket says 172.21.15.117:42140) 2020-11-20T13:56:55.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f3327fff700 1 -- 172.21.15.117:0/2439366634 learned_addr learned my addr 172.21.15.117:0/2439366634 (peer_addr_for_me v2:172.21.15.117:0/0) 2020-11-20T13:56:55.179 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332d16a700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332814a830 0x7f3328156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.158+0000 7f332c969700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f332805cd80 0x7f332815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3327fff700 1 -- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332814a830 msgr2=0x7f3328156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3327fff700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332814a830 0x7f3328156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.180 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3327fff700 1 -- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f332805cd80 msgr2=0x7f332815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3327fff700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f332805cd80 0x7f332815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3327fff700 1 -- 172.21.15.117:0/2439366634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332809d6e0 con 0x7f3328141260 2020-11-20T13:56:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f332c969700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f332805cd80 0x7f332815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:55.181 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f332d16a700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332814a830 0x7f3328156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T13:56:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3327fff700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328141260 0x7f332813fd50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f3314002f30 tx=0x7f3314015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3325ffb700 1 -- 172.21.15.117:0/2439366634 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f331401e460 con 0x7f3328141260 2020-11-20T13:56:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3325ffb700 1 -- 172.21.15.117:0/2439366634 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331401e5c0 con 0x7f3328141260 2020-11-20T13:56:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3328064a60 con 0x7f3328141260 2020-11-20T13:56:55.182 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f3325ffb700 1 -- 172.21.15.117:0/2439366634 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f331401e920 con 0x7f3328141260 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.159+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332809db80 con 0x7f3328141260 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.160+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3328057050 con 0x7f3328141260 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.160+0000 7f3325ffb700 1 -- 172.21.15.117:0/2439366634 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 63346+0+0 (secure 0 0 0) 0x7f3314025030 con 0x7f3328141260 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.160+0000 7f3325ffb700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3310051210 0x7f33100536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T13:56:55.183 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.160+0000 7f3325ffb700 1 -- 172.21.15.117:0/2439366634 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f331406a760 con 0x7f3328141260 2020-11-20T13:56:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.160+0000 7f332c969700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3310051210 0x7f33100536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T13:56:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.161+0000 7f332c969700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3310051210 0x7f33100536d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f331c00b000 tx=0x7f331c008040).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T13:56:55.184 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.164+0000 7f3325ffb700 1 -- 172.21.15.117:0/2439366634 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+133711 (secure 0 0 0) 0x7f3314042860 con 0x7f3328141260 2020-11-20T13:56:55.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.372+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f332809d870 con 0x7f3328141260 2020-11-20T13:56:55.378 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.373+0000 7f3325ffb700 1 -- 172.21.15.117:0/2439366634 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f3314023020 con 0x7f3328141260 2020-11-20T13:56:55.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3310051210 msgr2=0x7f33100536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3310051210 0x7f33100536d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f331c00b000 tx=0x7f331c008040).stop 2020-11-20T13:56:55.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328141260 msgr2=0x7f332813fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T13:56:55.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328141260 0x7f332813fd50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f3314002f30 tx=0x7f3314015000).stop 2020-11-20T13:56:55.379 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 shutdown_connections 2020-11-20T13:56:55.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:6800/22275,v1:172.21.15.117:6801/22275] conn(0x7f3310051210 0x7f33100536d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f332814a830 0x7f3328156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f332805cd80 0x7f332815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.380 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 --2- 172.21.15.117:0/2439366634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3328141260 0x7f332813fd50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T13:56:55.385 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 >> 172.21.15.117:0/2439366634 conn(0x7f33280b7480 msgr2=0x7f3328146f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T13:56:55.386 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 shutdown_connections 2020-11-20T13:56:55.386 INFO:teuthology.orchestra.run.smithi117.stderr:2020-11-20T13:56:55.374+0000 7f332ebcd700 1 -- 172.21.15.117:0/2439366634 wait complete. 2020-11-20T13:56:55.386 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 16 2020-11-20T13:56:55.386 INFO:teuthology.orchestra.run.smithi117.stdout: 2020-11-20T13:56:55.386 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5022,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6833/1931748750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6832","nonce":1931748750},{"type":"v1","addr":"172.21.15.85:6833","nonce":1931748750}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T13:56:33.416903+0000","modified":"2020-11-20T13:56:52.226497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6835/3369439478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3369439478},{"type":"v1","addr":"172.21.15.117:6835","nonce":3369439478}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T13:56:55.592 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T13:56:33.416903+0000', 'modified': '2020-11-20T13:56:52.226497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5242': {'gid': 5242, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6835/3369439478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3369439478}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 3369439478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T13:56:55.592 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T13:56:55.593 INFO:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T13:56:55.719 INFO:teuthology.orchestra.run.smithi117.stdout:{"cluster_fsid":"a1f0335b-4805-46c3-ba38-df4006f55611","whoami":0,"id":5242,"want_state":"up:active","state":"up:active","rank_uptime":6.4872044960000004,"mdsmap_epoch":16,"osdmap_epoch":37,"osdmap_epoch_barrier":37,"uptime":7.4025860699999999} 2020-11-20T13:56:55.720 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "a1f0335b-4805-46c3-ba38-df4006f55611", "id": 5242, "mdsmap_epoch": 16, "osdmap_epoch": 37, "osdmap_epoch_barrier": 37, "rank_uptime": 6.487204496, "state": "up:active", "uptime": 7.40258607, "want_state": "up:active", "whoami": 0 } 2020-11-20T13:56:55.720 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-11-20T13:56:55.721 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2020-11-20T13:56:55.721 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi117.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2020-11-20T13:56:55.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:55.722 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:55.737 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:55.811 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:55.812 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:55.878 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:55.879 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:55.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:55.880 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:55.933 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-11-20T13:56:55.934 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:55.975 INFO:teuthology.orchestra.run.smithi117:> sudo modprobe fuse 2020-11-20T13:56:56.005 INFO:teuthology.orchestra.run:Running command with timeout 30 2020-11-20T13:56:56.006 INFO:teuthology.orchestra.run.smithi117:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-11-20T13:56:56.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:2020-11-20T13:56:56.051+0000 7f16068c01c0 -1 init, newargv = 0x55f1e16dc240 newargc=9 2020-11-20T13:56:56.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[31607]: starting ceph client 2020-11-20T13:56:56.057 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi117.stderr:ceph-fuse[31607]: starting fuse 2020-11-20T13:56:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted on /sys/fs/fuse/connections. 2020-11-20T13:56:56.075 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-11-20T13:56:56.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:56.076 INFO:teuthology.orchestra.run.smithi117:> ls /sys/fs/fuse/connections 2020-11-20T13:56:56.129 INFO:teuthology.orchestra.run.smithi117.stdout:47 2020-11-20T13:56:56.129 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-11-20T13:56:56.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:56.130 INFO:teuthology.orchestra.run.smithi117:> sudo python3 -c ' 2020-11-20T13:56:56.131 INFO:teuthology.orchestra.run.smithi117:> import glob 2020-11-20T13:56:56.131 INFO:teuthology.orchestra.run.smithi117:> import re 2020-11-20T13:56:56.131 INFO:teuthology.orchestra.run.smithi117:> import os 2020-11-20T13:56:56.132 INFO:teuthology.orchestra.run.smithi117:> import subprocess 2020-11-20T13:56:56.132 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:56.132 INFO:teuthology.orchestra.run.smithi117:> def find_socket(client_name): 2020-11-20T13:56:56.133 INFO:teuthology.orchestra.run.smithi117:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2020-11-20T13:56:56.133 INFO:teuthology.orchestra.run.smithi117:> files = glob.glob(asok_path) 2020-11-20T13:56:56.133 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:56.134 INFO:teuthology.orchestra.run.smithi117:> # Given a non-glob path, it better be there 2020-11-20T13:56:56.134 INFO:teuthology.orchestra.run.smithi117:> if "*" not in asok_path: 2020-11-20T13:56:56.135 INFO:teuthology.orchestra.run.smithi117:> assert(len(files) == 1) 2020-11-20T13:56:56.135 INFO:teuthology.orchestra.run.smithi117:> return files[0] 2020-11-20T13:56:56.135 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:56.136 INFO:teuthology.orchestra.run.smithi117:> for f in files: 2020-11-20T13:56:56.141 INFO:teuthology.orchestra.run.smithi117:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-11-20T13:56:56.142 INFO:teuthology.orchestra.run.smithi117:> if os.path.exists("/proc/{0}".format(pid)): 2020-11-20T13:56:56.142 INFO:teuthology.orchestra.run.smithi117:> return f 2020-11-20T13:56:56.143 INFO:teuthology.orchestra.run.smithi117:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-11-20T13:56:56.143 INFO:teuthology.orchestra.run.smithi117:> 2020-11-20T13:56:56.143 INFO:teuthology.orchestra.run.smithi117:> print(find_socket("client.0")) 2020-11-20T13:56:56.144 INFO:teuthology.orchestra.run.smithi117:> ' 2020-11-20T13:56:56.222 INFO:teuthology.orchestra.run.smithi117.stdout:/var/run/ceph/ceph-client.0.31607.asok 2020-11-20T13:56:56.229 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31607.asok 2020-11-20T13:56:56.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:56.230 INFO:teuthology.orchestra.run.smithi117:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31607.asok status 2020-11-20T13:56:56.337 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2020-11-20T13:56:56.337 INFO:teuthology.orchestra.run.smithi117.stdout: "metadata": { 2020-11-20T13:56:56.337 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_sha1": "a7b62340ad27b26642bb54ec916a97c7e1f07a34", 2020-11-20T13:56:56.338 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 15.2.5-588-ga7b62340 (a7b62340ad27b26642bb54ec916a97c7e1f07a34) octopus (stable)", 2020-11-20T13:56:56.338 INFO:teuthology.orchestra.run.smithi117.stdout: "entity_id": "0", 2020-11-20T13:56:56.338 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": "smithi117", 2020-11-20T13:56:56.338 INFO:teuthology.orchestra.run.smithi117.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2020-11-20T13:56:56.338 INFO:teuthology.orchestra.run.smithi117.stdout: "pid": "31607", 2020-11-20T13:56:56.339 INFO:teuthology.orchestra.run.smithi117.stdout: "root": "/" 2020-11-20T13:56:56.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:56.339 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_count": 0, 2020-11-20T13:56:56.339 INFO:teuthology.orchestra.run.smithi117.stdout: "dentry_pinned_count": 0, 2020-11-20T13:56:56.340 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 5344, 2020-11-20T13:56:56.340 INFO:teuthology.orchestra.run.smithi117.stdout: "inst": { 2020-11-20T13:56:56.340 INFO:teuthology.orchestra.run.smithi117.stdout: "name": { 2020-11-20T13:56:56.340 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client", 2020-11-20T13:56:56.340 INFO:teuthology.orchestra.run.smithi117.stdout: "num": 5344 2020-11-20T13:56:56.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:56.342 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:56.342 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:56.342 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:56.343 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2846402987 2020-11-20T13:56:56.343 INFO:teuthology.orchestra.run.smithi117.stdout: } 2020-11-20T13:56:56.343 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:56.343 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": { 2020-11-20T13:56:56.343 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2020-11-20T13:56:56.344 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2020-11-20T13:56:56.344 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2846402987 2020-11-20T13:56:56.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2020-11-20T13:56:56.344 INFO:teuthology.orchestra.run.smithi117.stdout: "inst_str": "client.5344 172.21.15.117:0/2846402987", 2020-11-20T13:56:56.344 INFO:teuthology.orchestra.run.smithi117.stdout: "addr_str": "172.21.15.117:0/2846402987", 2020-11-20T13:56:56.345 INFO:teuthology.orchestra.run.smithi117.stdout: "inode_count": 1, 2020-11-20T13:56:56.345 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_epoch": 16, 2020-11-20T13:56:56.345 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch": 37, 2020-11-20T13:56:56.345 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_epoch_barrier": 0, 2020-11-20T13:56:56.345 INFO:teuthology.orchestra.run.smithi117.stdout: "blacklisted": false 2020-11-20T13:56:56.346 INFO:teuthology.orchestra.run.smithi117.stdout:} 2020-11-20T13:56:56.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:56.347 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2020-11-20T13:56:56.348 INFO:teuthology.orchestra.run.smithi117:> ' -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:56.363 INFO:teuthology.orchestra.run.smithi117.stdout:fuseblk 2020-11-20T13:56:56.363 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2020-11-20T13:56:56.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T13:56:56.364 INFO:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2020-11-20T13:56:56.431 INFO:teuthology.orchestra.run.smithi117:> cd /home/ubuntu/cephtest/mnt.0 && sudo ls testdir1