2020-03-19T22:56:53.660 INFO:root:teuthology version: 1.0.0-293e8c9 2020-03-19T22:56:53.661 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.QLVl6w.tmp -- --verbose --lock --description rados/singleton-nomsgr/{all/cache-fs-trunc.yaml mon_election/classic.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} --name sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi --block --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731 2020-03-19T22:56:53.693 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-19T22:56:53.754 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731 branch: wip-sage-testing-2020-03-19-0855 description: rados/singleton-nomsgr/{all/cache-fs-trunc.yaml mon_election/classic.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}} email: null first_in_suite: false job_id: '4868731' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: rhel os_version: '8.1' overrides: admin_socket: branch: wip-sage-testing-2020-03-19-0855 ceph: conf: global: mon election default strategy: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd debug verify cached snaps: true osd debug verify missing on start: true osd op queue: debug_random osd op queue cut off: debug_random log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 787db24c312ed9d310cd3f194b00548ede9a199c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 787db24c312ed9d310cd3f194b00548ede9a199c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-sage-testing-2020-03-19-0855 sha1: 787db24c312ed9d310cd3f194b00548ede9a199c owner: scheduled_sage@teuthology priority: 90 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - client.0 - client.1 sha1: 787db24c312ed9d310cd3f194b00548ede9a199c suite: rados suite_branch: wip-sage-testing-2020-03-19-0855 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 787db24c312ed9d310cd3f194b00548ede9a199c tasks: - install: null - ceph: conf: global: debug client: 20 debug mds: 20 debug ms: 1 osd max object name len: 460 osd max object namespace len: 64 log-whitelist: - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) pre-mgr-commands: - sudo ceph config set mgr mgr/devicehealth/enable_monitoring false --force - exec: client.0: - ceph osd pool create data_cache 4 - ceph osd tier add cephfs_data data_cache - ceph osd tier cache-mode data_cache writeback - ceph osd tier set-overlay cephfs_data data_cache - ceph osd pool set data_cache hit_set_type bloom - ceph osd pool set data_cache hit_set_count 8 - ceph osd pool set data_cache hit_set_period 3600 - ceph osd pool set data_cache min_read_recency_for_promote 0 - ceph-fuse: null - exec: client.0: - sudo chmod 777 $TESTDIR/mnt.0/ - dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5 - ls -al $TESTDIR/mnt.0/foo - truncate --size 0 $TESTDIR/mnt.0/foo - ls -al $TESTDIR/mnt.0/foo - dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5 - ls -al $TESTDIR/mnt.0/foo - cp $TESTDIR/mnt.0/foo /tmp/foo - sync - rados -p data_cache ls - - sleep 10 - rados -p data_cache ls - - rados -p data_cache cache-flush-evict-all - rados -p data_cache ls - - sleep 1 - exec: client.1: - hexdump -C /tmp/foo | head - hexdump -C $TESTDIR/mnt.1/foo | head - cmp $TESTDIR/mnt.1/foo /tmp/foo teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.16490 2020-03-19T22:56:53.754 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa; will attempt to use it 2020-03-19T22:56:53.755 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks 2020-03-19T22:56:53.755 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-03-19T22:56:53.756 INFO:teuthology.task.internal:Checking packages... 2020-03-19T22:56:53.769 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'basic' and ceph hash '787db24c312ed9d310cd3f194b00548ede9a199c' 2020-03-19T22:56:53.770 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-03-19T22:56:53.770 INFO:teuthology.packaging:ref: None 2020-03-19T22:56:53.770 INFO:teuthology.packaging:tag: None 2020-03-19T22:56:53.770 INFO:teuthology.packaging:branch: wip-sage-testing-2020-03-19-0855 2020-03-19T22:56:53.770 INFO:teuthology.packaging:sha1: 787db24c312ed9d310cd3f194b00548ede9a199c 2020-03-19T22:56:53.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-sage-testing-2020-03-19-0855 2020-03-19T22:56:53.987 INFO:teuthology.task.internal:Found packages for ceph version 15.1.1-234.g787db24 2020-03-19T22:56:53.987 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-03-19T22:56:53.997 INFO:teuthology.task.internal:no buildpackages task found 2020-03-19T22:56:53.997 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-03-19T22:56:54.012 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-03-19T22:56:54.013 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-19T22:56:54.058 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:57:04.074 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:57:14.091 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:57:24.107 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:57:34.124 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:57:44.141 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:57:54.159 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:58:04.177 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:58:14.194 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:58:24.210 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:58:34.227 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:58:44.243 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2020-03-19T22:58:54.258 DEBUG:teuthology.lock.ops:lock_many request: {'count': 1, 'locked_by': 'scheduled_sage@teuthology', 'description': '/home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731', 'machine_type': 'smithi'} 2020-03-19T22:58:54.276 DEBUG:teuthology.lock.ops:locked ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T22:58:54.303 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-19T22:58:54.523 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of rhel 8.1 2020-03-19T22:58:54.847 INFO:teuthology.orchestra.console:Power off smithi038 2020-03-19T22:58:54.848 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-19T22:58:54.870 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T22:58:58.893 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T22:59:03.003 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T22:59:03.214 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2020-03-19T22:59:03.315 INFO:teuthology.orchestra.console:Power on smithi038 2020-03-19T22:59:03.316 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-19T22:59:03.336 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T22:59:07.358 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T22:59:11.470 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-19T22:59:11.678 INFO:teuthology.orchestra.console:Power on for smithi038 completed 2020-03-19T22:59:11.779 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish 2020-03-19T23:01:42.868 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:01:51.930 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:02:00.991 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:02:10.075 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:02:19.126 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:02:28.191 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:02:37.274 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:03:43.276 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:03:43.539 INFO:teuthology.orchestra.run.smithi038:> hostname 2020-03-19T23:03:43.836 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com 2020-03-19T23:03:43.836 INFO:teuthology.orchestra.run.smithi038:> grep smithi038.front.sepia.ceph.com /etc/hosts 2020-03-19T23:03:43.852 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038 2020-03-19T23:03:43.852 INFO:teuthology.orchestra.run.smithi038:> sudo hostname smithi038 2020-03-19T23:03:43.981 INFO:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hosts 2020-03-19T23:03:44.014 INFO:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/172.21.15.38/172.21.15.38/g' /etc/hosts 2020-03-19T23:03:44.085 INFO:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hostname 2020-03-19T23:03:44.155 INFO:teuthology.orchestra.run.smithi038:> sudo hostname smithi038 2020-03-19T23:03:44.226 INFO:teuthology.provision.fog.smithi038:Deploy complete! 2020-03-19T23:03:44.228 DEBUG:teuthology.parallel:result is None 2020-03-19T23:03:44.334 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com 2020-03-19T23:03:44.335 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-19T23:03:44.355 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server 2020-03-19T23:03:44.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-19T23:03:44.356 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:03:44.441 INFO:teuthology.orchestra.run.smithi038:> true 2020-03-19T23:03:44.616 INFO:teuthology.orchestra.run.smithi038:> uname -m 2020-03-19T23:03:44.669 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2020-03-19T23:03:44.670 INFO:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2020-03-19T23:03:44.724 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Red Hat Enterprise Linux" 2020-03-19T23:03:44.724 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8.1 (Ootpa)" 2020-03-19T23:03:44.724 INFO:teuthology.orchestra.run.smithi038.stdout:ID="rhel" 2020-03-19T23:03:44.724 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="fedora" 2020-03-19T23:03:44.724 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8.1" 2020-03-19T23:03:44.724 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8" 2020-03-19T23:03:44.724 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-03-19T23:03:44.725 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2020-03-19T23:03:44.725 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-03-19T23:03:44.725 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.redhat.com/" 2020-03-19T23:03:44.725 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-03-19T23:03:44.725 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:03:44.725 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-03-19T23:03:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-03-19T23:03:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-03-19T23:03:44.726 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-03-19T23:03:44.727 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2020-03-19T23:03:44.740 INFO:teuthology.task.internal.lock_machines:1 smithi machines locked this try, 1/1 locked so far 2020-03-19T23:03:44.757 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXXrJ1MD0XA8HHO9piq5w48uyQLNujW4uBAQHZiAGFfP7tgFrzj5spaWB1hOIQNBperOdNaHJJ7Wtkx4tx4LJsHQsUwmqJzGXtuhulWyI3Bopr8elLzlhK22CTzhmOKF4GKIHiNrn/Aa/SwdQpuwslqGSeXkll44a+qI2VISu+JuQUlGnC5ENbWRPvpYUjGvapFlEcSoDVeZQQIBrWV+45ej0waj7cpzEyFsaNicXrixbNucm2hkbSOUYcqHba7JqkVqQZrfLRKcoRXV1c+87XoJaS47ZvToi8vUGsJRi10sdH4T+RU1oAJCXNoOKHXSDl46BwlU2zxXzEsn6pm8oR/7mziFAfF+PYIm6eKNdw1ka0c+KD3Y1E0PU5FcpCE8jjhXZWCQR34pGRO4m832oydY0cNSNy1cu1jNGdB8Vii5Qq1+v+YZyoar/t5/q30PIVxMXzHu3FkTqPvGUWh5IDo4BitrxdQRigPiVZk6hI8usXxbuZbfrrRnj6H4a3FNU= 2020-03-19T23:03:44.757 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-19T23:03:44.802 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-03-19T23:03:44.809 INFO:teuthology.task.internal:Saving configuration 2020-03-19T23:03:44.824 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-03-19T23:03:44.835 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-03-19T23:03:44.851 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-19 22:58:54.265135', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi038.front.sepia.ceph.com', u'os_version': u'8.1', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'rhel', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGucHWss3bV0Qe72NNMX4/0pzN0gJb08Q4/1uFZc8tFaDVLvGFhB9IqBBr5FG79iAkkBrtkTsHCXGJdtZho+TRA=', u'description': u'/home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731'} 2020-03-19T23:03:44.852 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-03-19T23:03:44.860 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1'] 2020-03-19T23:03:44.860 INFO:teuthology.run_tasks:Running task console_log... 2020-03-19T23:03:44.900 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-19T23:03:44.901 INFO:teuthology.run_tasks:Running task internal.connect... 2020-03-19T23:03:44.907 INFO:teuthology.task.internal:Opening connections... 2020-03-19T23:03:44.908 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:03:44.909 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:03:44.983 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-03-19T23:03:44.989 INFO:teuthology.orchestra.run.smithi038:> uname -m 2020-03-19T23:03:45.161 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2020-03-19T23:03:45.161 INFO:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2020-03-19T23:03:45.215 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Red Hat Enterprise Linux" 2020-03-19T23:03:45.215 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8.1 (Ootpa)" 2020-03-19T23:03:45.215 INFO:teuthology.orchestra.run.smithi038.stdout:ID="rhel" 2020-03-19T23:03:45.215 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="fedora" 2020-03-19T23:03:45.215 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8.1" 2020-03-19T23:03:45.215 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8" 2020-03-19T23:03:45.216 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-03-19T23:03:45.216 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2020-03-19T23:03:45.216 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-03-19T23:03:45.216 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.redhat.com/" 2020-03-19T23:03:45.216 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-03-19T23:03:45.216 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:03:45.216 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-03-19T23:03:45.217 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-03-19T23:03:45.217 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-03-19T23:03:45.217 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-03-19T23:03:45.217 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2020-03-19T23:03:45.232 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-03-19T23:03:45.251 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-03-19T23:03:45.258 INFO:teuthology.task.internal:Checking for old test directory... 2020-03-19T23:03:45.259 INFO:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2020-03-19T23:03:45.272 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-03-19T23:03:45.281 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-03-19T23:03:45.281 INFO:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2020-03-19T23:03:45.328 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-03-19T23:03:45.353 INFO:teuthology.run_tasks:Running task kernel... 2020-03-19T23:03:45.369 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'mds.a': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}} 2020-03-19T23:03:45.369 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}}, timeout 300 2020-03-19T23:03:45.370 INFO:teuthology.orchestra.run.smithi038:> uname -r 2020-03-19T23:03:45.383 INFO:teuthology.orchestra.run.smithi038.stdout:4.18.0-147.5.1.el8_1.x86_64 2020-03-19T23:03:45.383 INFO:teuthology.task.kernel:Running kernel on smithi038: 4.18.0-147.5.1.el8_1.x86_64 2020-03-19T23:03:45.384 INFO:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2020-03-19T23:03:47.144 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:03:47.145 INFO:teuthology.orchestra.run.smithi038.stdout:Unable to read consumer identity 2020-03-19T23:03:47.279 INFO:teuthology.orchestra.run.smithi038.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-03-19T23:03:48.348 INFO:teuthology.orchestra.run.smithi038.stdout:Copr repo for ceph-el8 owned by ktdreyer 291 kB/s | 38 kB 00:00 2020-03-19T23:03:48.622 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 25 MB/s | 6.1 MB 00:00 2020-03-19T23:03:49.870 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 5.6 MB/s | 30 kB 00:00 2020-03-19T23:03:50.143 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:03:50.143 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:03:50.143 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:03:50.143 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:03:50.354 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-03-19T23:03:50.355 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-4.18.0-147.5.1.el8_1.x86_64 is already installed. 2020-03-19T23:03:50.436 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:03:50.437 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:03:50.437 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:03:50.595 INFO:teuthology.orchestra.run.smithi038:> echo no | sudo yum reinstall kernel || true 2020-03-19T23:03:50.946 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:03:50.946 INFO:teuthology.orchestra.run.smithi038.stdout:Unable to read consumer identity 2020-03-19T23:03:51.041 INFO:teuthology.orchestra.run.smithi038.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-03-19T23:03:51.197 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:02 ago on Thu 19 Mar 2020 11:03:49 PM UTC. 2020-03-19T23:03:51.225 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:03:51.226 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:03:51.226 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:03:51.226 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:03:51.295 INFO:teuthology.orchestra.run.smithi038.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-03-19T23:03:51.295 INFO:teuthology.orchestra.run.smithi038.stdout:Installed package kernel-4.18.0-147.5.1.el8_1.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-03-19T23:03:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:Error: No packages marked for reinstall. 2020-03-19T23:03:51.439 INFO:teuthology.orchestra.run.smithi038:> sudo yum reinstall -y kernel || true 2020-03-19T23:03:51.789 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:03:51.790 INFO:teuthology.orchestra.run.smithi038.stdout:Unable to read consumer identity 2020-03-19T23:03:51.960 INFO:teuthology.orchestra.run.smithi038.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-03-19T23:03:52.118 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:03 ago on Thu 19 Mar 2020 11:03:49 PM UTC. 2020-03-19T23:03:52.148 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:03:52.148 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:03:52.149 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:03:52.149 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:03:52.223 INFO:teuthology.orchestra.run.smithi038.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-03-19T23:03:52.223 INFO:teuthology.orchestra.run.smithi038.stdout:Installed package kernel-4.18.0-147.5.1.el8_1.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-03-19T23:03:52.295 INFO:teuthology.orchestra.run.smithi038.stderr:Error: No packages marked for reinstall. 2020-03-19T23:03:52.355 INFO:teuthology.orchestra.run.smithi038:> rpm -q kernel --last | head -n 1 2020-03-19T23:03:52.409 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-4.18.0-147.5.1.el8_1.x86_64 Mon 16 Mar 2020 03:25:53 PM UTC 2020-03-19T23:03:52.410 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-147.5.1.el8_1.x86_64 2020-03-19T23:03:52.410 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-03-19T23:03:52.411 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-03-19T23:03:52.411 INFO:teuthology.orchestra.run.smithi038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-03-19T23:03:52.446 INFO:teuthology.orchestra.run.smithi038.stdout:ttyS1 2020-03-19T23:03:52.457 INFO:teuthology.run_tasks:Running task internal.base... 2020-03-19T23:03:52.467 INFO:teuthology.task.internal:Creating test directory... 2020-03-19T23:03:52.467 INFO:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-03-19T23:03:52.512 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-03-19T23:03:52.521 INFO:teuthology.run_tasks:Running task internal.archive... 2020-03-19T23:03:52.534 INFO:teuthology.task.internal:Creating archive directory... 2020-03-19T23:03:52.534 INFO:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-03-19T23:03:52.576 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-03-19T23:03:52.585 INFO:teuthology.task.internal:Enabling coredump saving... 2020-03-19T23:03:52.586 INFO:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-03-19T23:03:52.686 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-03-19T23:03:52.702 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-03-19T23:03:52.704 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-03-19T23:03:52.714 INFO:teuthology.task.internal:Configuring sudo... 2020-03-19T23:03:52.714 INFO:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-03-19T23:03:52.745 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-03-19T23:03:52.760 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-03-19T23:03:52.761 INFO:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-03-19T23:03:52.800 INFO:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-03-19T23:03:52.872 INFO:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-03-19T23:03:52.939 INFO:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-03-19T23:03:53.013 INFO:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-03-19T23:03:53.066 INFO:teuthology.orchestra.run.smithi038:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-03-19T23:03:53.152 INFO:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2020-03-19T23:03:53.252 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-03-19T23:03:53.644 INFO:teuthology.run_tasks:Running task internal.timer... 2020-03-19T23:03:53.659 INFO:teuthology.task.internal:Starting timer... 2020-03-19T23:03:53.659 INFO:teuthology.run_tasks:Running task pcp... 2020-03-19T23:03:53.691 INFO:teuthology.run_tasks:Running task selinux... 2020-03-19T23:03:53.713 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-03-19T23:03:53.713 INFO:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2020-03-19T23:03:53.778 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: [ OK ] 2020-03-19T23:03:53.780 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-03-19T23:03:53.780 INFO:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2020-03-19T23:03:53.800 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2020-03-19T23:03:53.801 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive'} 2020-03-19T23:03:53.801 INFO:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-03-19T23:03:53.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T23:03:53.872 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-03-19T23:03:53.873 INFO:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2020-03-19T23:03:53.954 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-03-19T23:03:53.966 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-03-19T23:03:53.966 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-03-19T23:03:53.987 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-03-19T23:03:53.990 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-03-19T23:03:54.344 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-03-19T23:03:55.506 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-03-19T23:03:58.624 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-03-19T23:03:58.647 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:03:58 +0000 (0:00:00.062) 0:00:00.062 ******** 2020-03-19T23:04:01.155 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi038.front.sepia.ceph.com] 2020-03-19T23:04:01.233 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:01 +0000 (0:00:02.586) 0:00:02.648 ******** 2020-03-19T23:04:01.593 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-03-19T23:04:01.595 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-03-19T23:04:01.629 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:01 +0000 (0:00:00.395) 0:00:03.044 ******** 2020-03-19T23:04:02.205 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-03-19T23:04:02.205 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-03-19T23:04:02.244 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:02 +0000 (0:00:00.614) 0:00:03.658 ******** 2020-03-19T23:04:02.268 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-03-19T23:04:02.270 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:02.325 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:02 +0000 (0:00:00.080) 0:00:03.740 ******** 2020-03-19T23:04:02.671 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-03-19T23:04:02.673 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T23:04:02.710 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:02 +0000 (0:00:00.385) 0:00:04.125 ******** 2020-03-19T23:04:03.711 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-03-19T23:04:03.712 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1584659042.73-15202609495853/source", "state": "file", "uid": 0} 2020-03-19T23:04:03.755 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:03 +0000 (0:00:01.045) 0:00:05.170 ******** 2020-03-19T23:04:04.339 INFO:teuthology.task.ansible.out: 2020-03-19T23:04:04.341 INFO:teuthology.task.ansible.out:TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCto 2020-03-19T23:04:04.342 INFO:teuthology.task.ansible.out:HDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-03-19T23:04:04.447 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-03-19T23:04:04.456 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-03-19T23:04:04.649 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:04 +0000 (0:00:00.894) 0:00:06.064 ******** 2020-03-19T23:04:05.350 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi038.front.sepia.ceph.com] 2020-03-19T23:04:05.425 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-03-19T23:04:05.426 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:05 +0000 (0:00:00.775) 0:00:06.840 ******** 2020-03-19T23:04:05.925 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-03-19T23:04:05.927 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.214617", "end": "2020-03-19 23:04:05.904776", "rc": 0, "start": "2020-03-19 23:04:05.690159", "stderr": "", "stderr_lines": [], "stdout": "b22ad6dbd27eb2f944823eb8be286527dd352386", "stdout_lines": ["b22ad6dbd27eb2f944823eb8be286527dd352386"]} 2020-03-19T23:04:05.966 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:05 +0000 (0:00:00.541) 0:00:07.381 ******** 2020-03-19T23:04:06.200 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-03-19T23:04:06.201 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.011850", "end": "2020-03-19 23:04:06.173481", "failed_when_result": false, "rc": 0, "start": "2020-03-19 23:04:06.161631", "stderr": "", "stderr_lines": [], "stdout": "b22ad6dbd27eb2f944823eb8be286527dd352386", "stdout_lines": ["b22ad6dbd27eb2f944823eb8be286527dd352386"]} 2020-03-19T23:04:06.241 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:06 +0000 (0:00:00.274) 0:00:07.656 ******** 2020-03-19T23:04:06.266 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-03-19T23:04:06.267 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-03-19T23:04:06.303 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:06 +0000 (0:00:00.062) 0:00:07.718 ******** 2020-03-19T23:04:06.331 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-03-19T23:04:06.332 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.375 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:06 +0000 (0:00:00.071) 0:00:07.790 ******** 2020-03-19T23:04:06.403 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-03-19T23:04:06.405 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.448 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:06 +0000 (0:00:00.073) 0:00:07.863 ******** 2020-03-19T23:04:06.473 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-03-19T23:04:06.475 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.522 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:06 +0000 (0:00:00.073) 0:00:07.937 ******** 2020-03-19T23:04:06.549 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-03-19T23:04:06.549 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.589 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:06 +0000 (0:00:00.066) 0:00:08.004 ******** 2020-03-19T23:04:06.616 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-03-19T23:04:06.617 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.677 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:06 +0000 (0:00:00.087) 0:00:08.092 ******** 2020-03-19T23:04:06.723 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-03-19T23:04:06.724 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.735 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.735 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.747 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.749 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.760 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.761 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.772 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.774 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.806 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.807 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.808 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOt 2020-03-19T23:04:06.810 INFO:teuthology.task.ansible.out:sItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc09659 2020-03-19T23:04:06.811 INFO:teuthology.task.ansible.out:4ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.821 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.822 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.834 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.845 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.861 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.872 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.874 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.885 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.886 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.905 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.909 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.910 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.920 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.921 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.932 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.932 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.943 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.944 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.955 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.955 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.977 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.978 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.979 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:06.990 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.001 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.012 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.023 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.036 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.045 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.056 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.057 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.068 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.079 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.079 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.091 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.092 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.103 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.104 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.115 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.127 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.128 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.139 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.140 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.151 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.152 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.163 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.164 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.175 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.187 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.188 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.199 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.199 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.210 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.211 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.222 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.223 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.235 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.236 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.237 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.248 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.260 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.260 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.272 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.273 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.284 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.295 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.296 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.307 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.308 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.319 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.330 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon BgvhtHMrCcelT98ekPDMoA 6fa3cf08186b93b966b68841f3507a0809d348c5097b39b6140832cac2afd511"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.330 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.341 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.341 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.343 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.353 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.366 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.379 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.382 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.384 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.394 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.395 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.407 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.407 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.419 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.431 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.442 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d', u'name': u'xiubli'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "xiubli@fedora1 vosB15bA3uALkBrmjcKFUg 4ef5cb132c30563fd5fb9e4e9352786ad0d2258e41ed63f91c1f65f69b54d50d"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.455 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f', u'name': u'jafaj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef', u'name': u'jschmid'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.466 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518', u'name': u'kachau'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad GsZIGCdLojpvOZDxwcWSlg 0638220db65953e0cd1f134c499e4a713b32a882c8a92458afcb495dbd087518"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.467 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034', u'name': u'khiremat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.478 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'ovpn': u'kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0', u'name': u'kapandya'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad Uw1JHj0kPIqSd9iOJ68TNg 46badc3b5cd150bfb74daf4300c8329491000521738ba70455631aa4a47e42c0"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.539 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:07 +0000 (0:00:00.861) 0:00:08.954 ******** 2020-03-19T23:04:07.565 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-03-19T23:04:07.609 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:07 +0000 (0:00:00.069) 0:00:09.024 ******** 2020-03-19T23:04:07.639 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.705 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:07 +0000 (0:00:00.096) 0:00:09.120 ******** 2020-03-19T23:04:07.741 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-03-19T23:04:07.742 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.781 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:07 +0000 (0:00:00.075) 0:00:09.196 ******** 2020-03-19T23:04:07.807 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-03-19T23:04:07.808 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.854 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:07 +0000 (0:00:00.073) 0:00:09.269 ******** 2020-03-19T23:04:07.881 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-03-19T23:04:07.882 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:07.927 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:07 +0000 (0:00:00.072) 0:00:09.342 ******** 2020-03-19T23:04:07.955 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-03-19T23:04:07.957 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.005 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:08 +0000 (0:00:00.078) 0:00:09.420 ******** 2020-03-19T23:04:08.053 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-03-19T23:04:08.055 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.066 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.066 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.068 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.079 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.090 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.092 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.103 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.104 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.115 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.116 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.126 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.128 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.138 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.139 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.150 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.151 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.162 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.173 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.184 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.185 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.196 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.207 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.208 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.224 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.232 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.242 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.243 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.253 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.254 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.265 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.265 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.276 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.277 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.279 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.289 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.291 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.302 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-03-19T23:04:08.352 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:08 +0000 (0:00:00.347) 0:00:09.767 ******** 2020-03-19T23:04:09.176 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-03-19T23:04:09.176 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "87deb5c6e571edae0503a547e464e6fdb70a7a2c", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "state": "file", "uid": 0} 2020-03-19T23:04:09.224 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:09 +0000 (0:00:00.871) 0:00:10.639 ******** 2020-03-19T23:04:09.249 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-03-19T23:04:09.251 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => { "msg": "Host smithi038.front.sepia.ceph.com is running RedHat 8.1 (Ootpa)" } 2020-03-19T23:04:09.305 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:09 +0000 (0:00:00.081) 0:00:10.720 ******** 2020-03-19T23:04:09.363 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-03-19T23:04:09.364 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"], "rhsm_repos": ["rhel-8-for-x86_64-baseos-rpms", "rhel-8-for-x86_64-appstream-rpms"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"} 2020-03-19T23:04:09.402 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:09 +0000 (0:00:00.096) 0:00:10.817 ******** 2020-03-19T23:04:09.437 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-03-19T23:04:09.439 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:09.479 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:09 +0000 (0:00:00.077) 0:00:10.894 ******** 2020-03-19T23:04:09.795 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-03-19T23:04:09.796 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.090724", "end": "2020-03-19 23:04:09.769114", "rc": 0, "start": "2020-03-19 23:04:09.678390", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-03-19T23:04:09.835 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:09 +0000 (0:00:00.355) 0:00:11.250 ******** 2020-03-19T23:04:09.871 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-03-19T23:04:09.872 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:09.908 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:09 +0000 (0:00:00.072) 0:00:11.323 ******** 2020-03-19T23:04:09.944 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-03-19T23:04:09.945 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:09.983 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:09 +0000 (0:00:00.074) 0:00:11.398 ******** 2020-03-19T23:04:10.010 INFO:teuthology.task.ansible.out: 2020-03-19T23:04:10.011 INFO:teuthology.task.ansible.out:TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:10.058 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:10 +0000 (0:00:00.075) 0:00:11.473 ******** 2020-03-19T23:04:10.090 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-03-19T23:04:10.092 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:10.137 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:10 +0000 (0:00:00.079) 0:00:11.552 ******** 2020-03-19T23:04:10.174 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-03-19T23:04:10.176 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:10.231 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:10 +0000 (0:00:00.094) 0:00:11.646 ******** 2020-03-19T23:04:10.270 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-03-19T23:04:10.271 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"entitlements_path": "/etc/ansible/secrets/entitlements.yml"}, "changed": false} 2020-03-19T23:04:10.323 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:10 +0000 (0:00:00.091) 0:00:11.738 ******** 2020-03-19T23:04:10.369 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-03-19T23:04:10.370 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi038.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-03-19T23:04:10.434 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:10 +0000 (0:00:00.111) 0:00:11.849 ******** 2020-03-19T23:04:10.491 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-03-19T23:04:10.493 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"have_entitlements": true}, "changed": false} 2020-03-19T23:04:10.534 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:10 +0000 (0:00:00.100) 0:00:11.949 ******** 2020-03-19T23:04:11.804 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-03-19T23:04:11.806 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed /root/.ansible/tmp/ansible-moduletmp-1584659051.6822336-7_w0hh2c/katello-ca-consumer-latest.noarchre5gu3po.rpm"]} 2020-03-19T23:04:11.843 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:11 +0000 (0:00:01.308) 0:00:13.258 ******** 2020-03-19T23:04:12.832 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-03-19T23:04:12.833 INFO:teuthology.task.ansible.out:fatal: [smithi038.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ...ignoring 2020-03-19T23:04:12.868 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:12 +0000 (0:00:01.024) 0:00:14.283 ******** 2020-03-19T23:04:12.903 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-03-19T23:04:12.905 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": false}, "changed": false} 2020-03-19T23:04:12.941 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:12 +0000 (0:00:00.072) 0:00:14.355 ******** 2020-03-19T23:04:22.356 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** changed: [smithi038.front.sepia.ceph.com] => {"attempts": 1, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-03-19T23:04:22.393 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:22 +0000 (0:00:09.453) 0:00:23.808 ******** 2020-03-19T23:04:22.431 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-03-19T23:04:22.433 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": true}, "changed": false} 2020-03-19T23:04:22.477 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:22 +0000 (0:00:00.083) 0:00:23.892 ******** 2020-03-19T23:04:23.425 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-03-19T23:04:23.425 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager release --list | grep -E '[0-9]'", "delta": "0:00:00.732937", "end": "2020-03-19 23:04:23.400689", "failed_when_result": false, "rc": 0, "start": "2020-03-19 23:04:22.667752", "stderr": "", "stderr_lines": [], "stdout": "7.4\n7.5\n7.6\n7.7\n7Server\n8\n8.0\n8.1", "stdout_lines": ["7.4", "7.5", "7.6", "7.7", "7Server", "8", "8.0", "8.1"]} 2020-03-19T23:04:23.461 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:23 +0000 (0:00:00.984) 0:00:24.876 ******** 2020-03-19T23:04:23.497 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-03-19T23:04:23.498 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:23.535 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:23 +0000 (0:00:00.073) 0:00:24.949 ******** 2020-03-19T23:04:23.570 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-03-19T23:04:23.571 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:23.608 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:23 +0000 (0:00:00.073) 0:00:25.023 ******** 2020-03-19T23:04:23.635 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-03-19T23:04:23.639 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-03-19T23:04:23.672 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:23 +0000 (0:00:00.063) 0:00:25.087 ******** 2020-03-19T23:04:23.708 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-03-19T23:04:23.710 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:23.755 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:23 +0000 (0:00:00.082) 0:00:25.170 ******** 2020-03-19T23:04:34.200 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-03-19T23:04:34.201 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager repos --list | grep -B4 'Enabled:.*1' | grep 'Repo ID:' | sed -e 's/Repo ID:\\s*\\(.*\\)/\\1/' | sort", "delta": "0:00:10.217299", "end": "2020-03-19 23:04:34.173712", "rc": 0, "start": "2020-03-19 23:04:23.956413", "stderr": "", "stderr_lines": [], "stdout": "rhel-8-for-x86_64-appstream-rpms\nrhel-8-for-x86_64-baseos-rpms", "stdout_lines": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]} 2020-03-19T23:04:34.255 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:34 +0000 (0:00:10.500) 0:00:35.670 ******** 2020-03-19T23:04:34.312 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-03-19T23:04:34.313 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"repo_list": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]}, "changed": false} 2020-03-19T23:04:34.362 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:34 +0000 (0:00:00.106) 0:00:35.777 ******** 2020-03-19T23:04:34.408 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-03-19T23:04:34.409 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:34.447 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:34 +0000 (0:00:00.085) 0:00:35.862 ******** 2020-03-19T23:04:34.483 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-03-19T23:04:34.485 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": false}, "changed": false} 2020-03-19T23:04:34.523 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:34 +0000 (0:00:00.075) 0:00:35.938 ******** 2020-03-19T23:04:34.570 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-03-19T23:04:34.572 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2020-03-19T23:04:34.620 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:34 +0000 (0:00:00.096) 0:00:36.035 ******** 2020-03-19T23:04:40.400 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-03-19T23:04:40.401 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-03-19T23:04:40.441 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:40 +0000 (0:00:05.821) 0:00:41.856 ******** 2020-03-19T23:04:45.871 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-03-19T23:04:45.871 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item=rhel-8-for-x86_64-baseos-rpms) => {"ansible_loop_var": "item", "changed": true, "cmd": ["subscription-manager", "repos", "--enable", "rhel-8-for-x86_64-baseos-rpms"], "delta": "0:00:05.196379", "end": "2020-03-19 23:04:45.846229", "item": "rhel-8-for-x86_64-baseos-rpms", "rc": 0, "start": "2020-03-19 23:04:40.649850", "stderr": "", "stderr_lines": [], "stdout": "Repository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system.", "stdout_lines": ["Repository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system."]} 2020-03-19T23:04:51.537 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item=rhel-8-for-x86_64-appstream-rpms) => {"ansible_loop_var": "item", "changed": true, "cmd": ["subscription-manager", "repos", "--enable", "rhel-8-for-x86_64-appstream-rpms"], "delta": "0:00:05.465390", "end": "2020-03-19 23:04:51.513735", "item": "rhel-8-for-x86_64-appstream-rpms", "rc": 0, "start": "2020-03-19 23:04:46.048345", "stderr": "", "stderr_lines": [], "stdout": "Repository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system.", "stdout_lines": ["Repository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system."]} 2020-03-19T23:04:51.577 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:51 +0000 (0:00:11.135) 0:00:52.992 ******** 2020-03-19T23:04:51.800 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-03-19T23:04:51.801 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "path": "/etc/yum.repos.d/cd.repo", "state": "absent"} 2020-03-19T23:04:51.847 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:51 +0000 (0:00:00.270) 0:00:53.262 ******** 2020-03-19T23:04:52.060 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-03-19T23:04:52.061 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T23:04:52.099 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:52 +0000 (0:00:00.252) 0:00:53.514 ******** 2020-03-19T23:04:52.630 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-03-19T23:04:52.630 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'enabled': u'1', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-03-19T23:04:53.116 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'enabled': 0, u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-03-19T23:04:53.156 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:53 +0000 (0:00:01.056) 0:00:54.571 ******** 2020-03-19T23:04:53.740 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-03-19T23:04:53.741 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-03-19T23:04:54.238 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-03-19T23:04:54.275 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:01.118) 0:00:55.690 ******** 2020-03-19T23:04:54.311 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.347 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.072) 0:00:55.762 ******** 2020-03-19T23:04:54.372 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-03-19T23:04:54.374 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.409 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.061) 0:00:55.824 ******** 2020-03-19T23:04:54.434 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-03-19T23:04:54.436 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.475 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.066) 0:00:55.890 ******** 2020-03-19T23:04:54.512 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-03-19T23:04:54.513 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.549 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.073) 0:00:55.964 ******** 2020-03-19T23:04:54.585 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-03-19T23:04:54.585 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.652 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.103) 0:00:56.067 ******** 2020-03-19T23:04:54.678 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-03-19T23:04:54.679 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.715 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.062) 0:00:56.130 ******** 2020-03-19T23:04:54.740 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-03-19T23:04:54.741 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.776 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.061) 0:00:56.191 ******** 2020-03-19T23:04:54.802 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-03-19T23:04:54.804 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.839 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.062) 0:00:56.254 ******** 2020-03-19T23:04:54.864 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-03-19T23:04:54.866 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:54.903 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:54 +0000 (0:00:00.063) 0:00:56.318 ******** 2020-03-19T23:04:55.518 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-03-19T23:04:55.520 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.414511", "end": "2020-03-19 23:04:55.500346", "rc": 0, "start": "2020-03-19 23:04:55.085835", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:04:55.561 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:55 +0000 (0:00:00.657) 0:00:56.976 ******** 2020-03-19T23:04:57.972 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-03-19T23:04:57.973 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-03-19T23:04:58.008 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:58 +0000 (0:00:02.447) 0:00:59.423 ******** 2020-03-19T23:04:58.033 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-03-19T23:04:58.035 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:58.070 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:58 +0000 (0:00:00.061) 0:00:59.485 ******** 2020-03-19T23:04:58.107 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-03-19T23:04:58.109 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:58.144 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:58 +0000 (0:00:00.073) 0:00:59.559 ******** 2020-03-19T23:04:58.169 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-03-19T23:04:58.171 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:58.222 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:58 +0000 (0:00:00.078) 0:00:59.637 ******** 2020-03-19T23:04:58.248 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-03-19T23:04:58.249 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:04:58.289 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:58 +0000 (0:00:00.066) 0:00:59.704 ******** 2020-03-19T23:04:58.789 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-03-19T23:04:58.790 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-03-19T23:04:58.834 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:04:58 +0000 (0:00:00.545) 0:01:00.249 ******** 2020-03-19T23:05:00.857 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-03-19T23:05:00.858 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: smartmontools"]} 2020-03-19T23:05:00.912 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:00 +0000 (0:00:02.077) 0:01:02.327 ******** 2020-03-19T23:05:01.477 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi038.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-03-19T23:05:01.975 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-03-19T23:05:02.432 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-03-19T23:05:02.468 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:02 +0000 (0:00:01.556) 0:01:03.883 ******** 2020-03-19T23:05:02.680 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-03-19T23:05:02.716 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:02 +0000 (0:00:00.247) 0:01:04.131 ******** 2020-03-19T23:05:03.218 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-03-19T23:05:03.219 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-03-19T23:05:03.729 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-03-19T23:05:04.187 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-03-19T23:05:04.236 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:04 +0000 (0:00:01.520) 0:01:05.651 ******** 2020-03-19T23:05:04.263 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-03-19T23:05:04.265 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:05:04.303 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:04 +0000 (0:00:00.066) 0:01:05.718 ******** 2020-03-19T23:05:04.341 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-03-19T23:05:04.343 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-03-19T23:05:04.377 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:04 +0000 (0:00:00.074) 0:01:05.792 ******** 2020-03-19T23:05:04.599 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi038.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-03-19T23:05:04.651 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:04 +0000 (0:00:00.273) 0:01:06.066 ******** 2020-03-19T23:05:04.701 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-03-19T23:05:04.703 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T23:05:04.713 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T23:05:04.726 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T23:05:04.738 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-03-19T23:05:04.787 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:04 +0000 (0:00:00.136) 0:01:06.202 ******** 2020-03-19T23:05:06.855 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-03-19T23:05:06.856 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2020-03-19T23:05:06.904 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:06 +0000 (0:00:02.116) 0:01:08.319 ******** 2020-03-19T23:05:06.940 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-03-19T23:05:06.941 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:05:06.977 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:06 +0000 (0:00:00.072) 0:01:08.392 ******** 2020-03-19T23:05:07.485 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-03-19T23:05:07.485 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-03-19T23:05:07.543 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:07 +0000 (0:00:00.566) 0:01:08.958 ******** 2020-03-19T23:05:08.100 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-03-19T23:05:08.100 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-03-19T23:05:08.138 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:08 +0000 (0:00:00.594) 0:01:09.553 ******** 2020-03-19T23:05:08.164 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-03-19T23:05:08.165 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:05:08.205 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:08 +0000 (0:00:00.067) 0:01:09.620 ******** 2020-03-19T23:05:08.439 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-03-19T23:05:08.441 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T23:05:08.513 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:08 +0000 (0:00:00.307) 0:01:09.928 ******** 2020-03-19T23:05:08.787 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-03-19T23:05:08.789 INFO:teuthology.task.ansible.out:fatal: [smithi038.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-03-19T23:05:08.827 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:08 +0000 (0:00:00.314) 0:01:10.242 ******** 2020-03-19T23:05:08.863 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-03-19T23:05:08.864 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:05:08.901 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:08 +0000 (0:00:00.073) 0:01:10.316 ******** 2020-03-19T23:05:09.433 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-03-19T23:05:09.435 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2020-03-19T23:05:09.481 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:09 +0000 (0:00:00.579) 0:01:10.896 ******** 2020-03-19T23:05:10.321 INFO:teuthology.task.ansible.out: 2020-03-19T23:05:10.323 INFO:teuthology.task.ansible.out:TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-03-19T23:05:10.327 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2020-03-19 23:03:10 UTC", "ActiveEnterTimestampMonotonic": "14385525", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount nss-lookup.target basic.target system.slice time-sync.target sysinit.target systemd-journald.socket systemd-tmpfiles-setup.service -.mount tmp.mount network.target local-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2020-03-19 23:03:10 UTC", "AssertTimestampMonotonic": "14384639", "Before": "multi-user.target getty@tty1.service xdm.service shutdown.target plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2020-03-19 23:03:10 UTC", "ConditionTimestampMonotonic": "14384639", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "D 2020-03-19T23:05:10.328 INFO:teuthology.task.ansible.out:efaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2412", "ExecMainStartTimestamp": "Thu 2020-03-19 23:03:10 UTC", "ExecMainStartTimestampMonotonic": "14385478", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[Thu 2020-03-19 23:03:10 UTC] ; stop_time=[n/a] ; pid=2412 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2020-03-19 23:03:10 UTC", "InactiveExitTimestampMonotonic": "14385525", "InvocationID": "1c0c0a928be145ed98c03297fe02a723", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "Limi 2020-03-19T23:05:10.331 INFO:teuthology.task.ansible.out:tDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127467", "LimitNPROCSoft": "127467", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127467", "LimitSIGPENDINGSoft": "127467", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2412", "MemoryAccounting": "yes", "MemoryCurrent": "1085440", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "n 2020-03-19T23:05:10.332 INFO:teuthology.task.ansible.out:rpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2020-03-19 23:03:10 UTC", "StateChangeTimestampMonotonic": "14385525", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2020-03-19 23:03:10 UTC", "WatchdogTimestampMonotonic": "14385523", "WatchdogUSec": "0"}} 2020-03-19T23:05:10.379 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:10 +0000 (0:00:00.898) 0:01:11.794 ******** 2020-03-19T23:05:10.618 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-03-19T23:05:10.618 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001810", "end": "2020-03-19 23:05:10.596628", "rc": 0, "start": "2020-03-19 23:05:10.594818", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-03-19T23:05:10.655 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:10 +0000 (0:00:00.275) 0:01:12.070 ******** 2020-03-19T23:05:12.748 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-03-19T23:05:12.749 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-03-19T23:05:12.785 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:12 +0000 (0:00:02.130) 0:01:14.200 ******** 2020-03-19T23:05:14.899 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-03-19T23:05:14.901 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: selinux-policy-targeted"]} 2020-03-19T23:05:14.941 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:14 +0000 (0:00:02.155) 0:01:16.356 ******** 2020-03-19T23:05:16.288 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-03-19T23:05:16.289 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-03-19T23:05:16.333 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:16 +0000 (0:00:01.391) 0:01:17.748 ******** 2020-03-19T23:05:33.443 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-03-19T23:05:33.444 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.886797", "end": "2020-03-19 23:05:33.421181", "failed_when_result": false, "rc": 0, "start": "2020-03-19 23:05:16.534384", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-03-19T23:05:33.482 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:33 +0000 (0:00:17.148) 0:01:34.897 ******** 2020-03-19T23:05:34.004 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-03-19T23:05:34.005 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-03-19T23:05:34.074 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:34 +0000 (0:00:00.591) 0:01:35.489 ******** 2020-03-19T23:05:34.298 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.011716", "end": "2020-03-19 23:05:34.268844", "rc": 0, "start": "2020-03-19 23:05:34.257128", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:05:34.347 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:34 +0000 (0:00:00.272) 0:01:35.762 ******** 2020-03-19T23:05:34.570 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-03-19T23:05:34.571 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.010495", "end": "2020-03-19 23:05:34.544956", "rc": 0, "start": "2020-03-19 23:05:34.534461", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:05:34.608 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:34 +0000 (0:00:00.261) 0:01:36.023 ******** 2020-03-19T23:05:51.634 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-03-19T23:05:51.636 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.812310", "end": "2020-03-19 23:05:51.609962", "rc": 0, "start": "2020-03-19 23:05:34.797652", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:05:51.676 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:51 +0000 (0:00:17.067) 0:01:53.091 ******** 2020-03-19T23:05:51.887 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-03-19T23:05:51.888 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-03-19T23:05:51.928 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:51 +0000 (0:00:00.252) 0:01:53.344 ******** 2020-03-19T23:05:51.965 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-03-19T23:05:51.966 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-03-19T23:05:52.003 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:51 +0000 (0:00:00.074) 0:01:53.418 ******** 2020-03-19T23:05:52.039 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-03-19T23:05:52.039 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-03-19T23:05:52.089 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:52 +0000 (0:00:00.085) 0:01:53.504 ******** 2020-03-19T23:05:52.125 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-03-19T23:05:52.126 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "copr_repos": ["ktdreyer/ceph-el8"], "epel_packages": ["dbench"], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["@core", "@base", "dnf-utils", "git-all", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "podman", "podman-docker", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "autoconf", "gdb"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"} 2020-03-19T23:05:52.170 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:52 +0000 (0:00:00.081) 0:01:53.585 ******** 2020-03-19T23:05:52.208 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-03-19T23:05:52.209 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-03-19T23:05:52.266 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:52 +0000 (0:00:00.096) 0:01:53.681 ******** 2020-03-19T23:05:52.477 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-03-19T23:05:52.527 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:52 +0000 (0:00:00.260) 0:01:53.942 ******** 2020-03-19T23:05:52.740 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-03-19T23:05:52.742 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "name": "ubuntu", "state": "present", "system": false} 2020-03-19T23:05:52.791 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:52 +0000 (0:00:00.264) 0:01:54.206 ******** 2020-03-19T23:05:52.829 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-03-19T23:05:52.830 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:05:52.892 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:52 +0000 (0:00:00.101) 0:01:54.307 ******** 2020-03-19T23:05:52.930 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-03-19T23:05:52.932 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:05:52.972 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:52 +0000 (0:00:00.079) 0:01:54.387 ******** 2020-03-19T23:05:53.196 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-03-19T23:05:53.197 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-03-19T23:05:53.260 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:53 +0000 (0:00:00.288) 0:01:54.675 ******** 2020-03-19T23:05:53.954 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-03-19T23:05:53.955 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-03-19T23:05:54.022 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:54 +0000 (0:00:00.758) 0:01:55.434 ******** 2020-03-19T23:05:54.055 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-03-19T23:05:54.057 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi038"}, "changed": false} 2020-03-19T23:05:54.092 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:54 +0000 (0:00:00.073) 0:01:55.507 ******** 2020-03-19T23:05:55.162 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-03-19T23:05:55.163 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi038", "ansible_hostname": "smithi038", "ansible_nodename": "smithi038"}, "changed": false, "name": "smithi038"} 2020-03-19T23:05:55.204 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:55 +0000 (0:00:01.111) 0:01:56.619 ******** 2020-03-19T23:05:55.704 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-03-19T23:05:55.705 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-03-19T23:05:55.742 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:55 +0000 (0:00:00.537) 0:01:57.157 ******** 2020-03-19T23:05:56.254 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-03-19T23:05:56.255 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-03-19T23:05:56.296 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:56 +0000 (0:00:00.553) 0:01:57.711 ******** 2020-03-19T23:05:57.017 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-03-19T23:05:57.017 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-03-19T23:05:57.085 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:57 +0000 (0:00:00.788) 0:01:58.500 ******** 2020-03-19T23:05:57.604 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-03-19T23:05:57.604 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-03-19T23:05:57.649 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:57 +0000 (0:00:00.564) 0:01:59.064 ******** 2020-03-19T23:05:57.864 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-03-19T23:05:57.864 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-03-19T23:05:57.902 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:57 +0000 (0:00:00.252) 0:01:59.317 ******** 2020-03-19T23:05:58.124 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-03-19T23:05:58.125 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-03-19T23:05:58.168 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:58 +0000 (0:00:00.266) 0:01:59.583 ******** 2020-03-19T23:05:58.404 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-03-19T23:05:58.404 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1211, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-03-19T23:05:58.441 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:58 +0000 (0:00:00.272) 0:01:59.856 ******** 2020-03-19T23:05:58.940 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-03-19T23:05:58.940 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-03-19T23:05:59.008 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:59 +0000 (0:00:00.567) 0:02:00.423 ******** 2020-03-19T23:05:59.536 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-03-19T23:05:59.536 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-03-19T23:05:59.576 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:05:59 +0000 (0:00:00.568) 0:02:00.991 ******** 2020-03-19T23:05:59.578 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:01.537 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-03-19T23:06:01.538 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-03-19T23:06:01.539 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.736103", "end": "2020-03-19 23:06:01.508654", "rc": 0, "start": "2020-03-19 23:05:59.772551", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:06:01.586 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:01 +0000 (0:00:02.009) 0:02:03.001 ******** 2020-03-19T23:06:01.626 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-03-19T23:06:01.627 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:01.678 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:01 +0000 (0:00:00.092) 0:02:03.093 ******** 2020-03-19T23:06:02.225 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-03-19T23:06:02.226 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-03-19T23:06:02.269 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:02 +0000 (0:00:00.590) 0:02:03.684 ******** 2020-03-19T23:06:02.296 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-03-19T23:06:02.355 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:02 +0000 (0:00:00.086) 0:02:03.770 ******** 2020-03-19T23:06:03.467 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-03-19T23:06:03.468 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item=ktdreyer/ceph-el8) => {"ansible_loop_var": "item", "changed": true, "cmd": ["dnf", "-y", "copr", "enable", "ktdreyer/ceph-el8"], "delta": "0:00:00.877945", "end": "2020-03-19 23:06:03.437553", "item": "ktdreyer/ceph-el8", "rc": 0, "start": "2020-03-19 23:06:02.559608", "stderr": "", "stderr_lines": [], "stdout": "Updating Subscription Management repositories.\nRepository successfully enabled.", "stdout_lines": ["Updating Subscription Management repositories.", "Repository successfully enabled."], "warnings": ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]} 2020-03-19T23:06:03.469 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2020-03-19T23:06:03.506 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:03 +0000 (0:00:01.151) 0:02:04.921 ******** 2020-03-19T23:06:03.532 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-03-19T23:06:03.534 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:03.580 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:03 +0000 (0:00:00.073) 0:02:04.995 ******** 2020-03-19T23:06:03.629 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-03-19T23:06:03.630 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:03.689 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:03 +0000 (0:00:00.109) 0:02:05.104 ******** 2020-03-19T23:06:03.716 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-03-19T23:06:03.718 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:03.755 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:03 +0000 (0:00:00.065) 0:02:05.170 ******** 2020-03-19T23:06:03.780 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T23:06:03.782 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:03.824 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:03 +0000 (0:00:00.068) 0:02:05.239 ******** 2020-03-19T23:06:03.864 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-03-19T23:06:03.865 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:03.900 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:03 +0000 (0:00:00.076) 0:02:05.315 ******** 2020-03-19T23:06:03.936 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-03-19T23:06:03.937 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:03.976 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:03 +0000 (0:00:00.075) 0:02:05.391 ******** 2020-03-19T23:06:04.198 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-03-19T23:06:04.199 INFO:teuthology.task.ansible.out:fatal: [smithi038.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.006969", "end": "2020-03-19 23:06:04.178490", "msg": "non-zero return code", "rc": 1, "start": "2020-03-19 23:06:04.171521", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-03-19T23:06:04.240 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:04 +0000 (0:00:00.263) 0:02:05.655 ******** 2020-03-19T23:06:04.265 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-03-19T23:06:04.266 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:04.301 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:04 +0000 (0:00:00.060) 0:02:05.716 ******** 2020-03-19T23:06:10.352 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64", "Removed: qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+3ff2765e.2.x86_64"]} 2020-03-19T23:06:10.394 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:10 +0000 (0:00:06.093) 0:02:11.809 ******** 2020-03-19T23:06:17.012 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:17.014 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure ceph dependency packages are not present.] ************* changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: tuned-2.12.0-3.el8_1.1.noarch", "Removed: hdparm-9.54-2.el8.x86_64", "Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2020-03-19T23:06:17.057 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:17 +0000 (0:00:06.657) 0:02:18.467 ******** 2020-03-19T23:06:28.416 INFO:teuthology.task.ansible.out: TASK [testnode : Base group workaround for BZ1782899] ************************** 2020-03-19T23:06:28.418 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": "dnf -y groupinstall base --nobest && dnf -y group upgrade base", "delta": "0:00:11.129814", "end": "2020-03-19 23:06:28.388079", "rc": 0, "start": "2020-03-19 23:06:17.258265", "stderr": "", "stderr_lines": [], "stdout": "Updating Subscription Management repositories.\nCopr repo for ceph-el8 owned by ktdreyer 66 kB/s | 3.3 kB 00:00 \nExtra Packages for Enterprise Linux 138 kB/s | 5.3 kB 00:00 \nlab-extras 1.2 MB/s | 2.9 kB 00:00 \nRed Hat Enterprise Linux 8 for x86_64 - AppStre 75 kB/s | 2.8 kB 00:00 \nRed Hat Enterprise Linux 8 for x86_64 - BaseOS 61 kB/s | 2.4 kB 00:00 \nDependencies resolved.\n================================================================================\n Package Architecture Version Repository Size\n================================================================================\nInstalling Groups:\n Base \n\nTransaction Summary\n================================================================================\n\nComplete!\nUpdating Subscription Management repositories.\nRed Hat Enterprise Linux 8 for x86_64 - AppStre 57 kB/s | 2.8 kB 00:00 \nRed Hat Enterprise Linux 8 for x86_64 - BaseOS 59 kB/s | 2.4 kB 00:00 \nDependencies resolved.\n================================================================================\n Package Architecture Version Repository Size\n================================================================================\nUpgrading Groups:\n Base \n\nTransaction Summary\n================================================================================\n\nComplete!", "stdout_lines": ["Updating Subscription Management repositories.", " 2020-03-19T23:06:28.419 INFO:teuthology.task.ansible.out:Copr repo for ceph-el8 owned by ktdreyer 66 kB/s | 3.3 kB 00:00 ", "Extra Packages for Enterprise Linux 138 kB/s | 5.3 kB 00:00 ", "lab-extras 1.2 MB/s | 2.9 kB 00:00 ", "Red Hat Enterprise Linux 8 for x86_64 - AppStre 75 kB/s | 2.8 kB 00:00 ", "Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 kB/s | 2.4 kB 00:00 ", "Dependencies resolved.", "================================================================================", " Package Architecture Version Repository Size", "================================================================================", "Installing Groups:", " Base ", "", "Transaction Summary", "================================================================================", "", "Complete!", "Updating Subscription Management repositories.", "Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 kB/s | 2.8 kB 00:00 ", "Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 kB/s | 2.4 kB 00:00 ", "Dependencies resolved.", "================================================================================", " Package Architecture Version Repository Size", "================================================================================", "Upgrading Groups:", " Base ", "", "Transaction Summary", "================================================================================", "", "Complete!"]} 2020-03-19T23:06:28.460 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:28 +0000 (0:00:11.407) 0:02:29.875 ******** 2020-03-19T23:06:40.348 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-03-19T23:06:40.349 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Group core installed.", "Group base installed.", "Installed: dnf-utils", "Installed: git-all", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: podman", "Installed: podman-docker", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: autoconf", "Installed: gdb", "Installed: boost-random-1.66.0-6.el8.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: tuned-2.12.0-3.el8_1.1.noarch", "Installed: boost-program-options-1.66.0-6.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+3ff2765e.2.x86_64", "Installed: hdparm-9.54-2.el8.x86_64"]} 2020-03-19T23:06:40.384 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:40 +0000 (0:00:11.924) 0:02:41.799 ******** 2020-03-19T23:06:42.653 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-03-19T23:06:42.655 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: dbench"]} 2020-03-19T23:06:42.692 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:42 +0000 (0:00:02.307) 0:02:44.107 ******** 2020-03-19T23:06:44.747 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-03-19T23:06:44.748 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-03-19T23:06:44.785 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:44 +0000 (0:00:02.092) 0:02:46.200 ******** 2020-03-19T23:06:44.821 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-03-19T23:06:44.822 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:44.858 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:44 +0000 (0:00:00.073) 0:02:46.273 ******** 2020-03-19T23:06:45.204 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-03-19T23:06:45.206 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service polkit.service sysinit.target dbus.socket system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ip6tables.service ebtables.service ipset.service shutdown.target iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-03-19T23:06:45.207 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127467", "LimitNPROCSoft": "127467", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127467", 2020-03-19T23:06:45.208 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127467", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket system.slice sysinit.target", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-03-19T23:06:45.210 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-03-19T23:06:45.243 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:45 +0000 (0:00:00.384) 0:02:46.658 ******** 2020-03-19T23:06:45.551 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-03-19T23:06:45.552 INFO:teuthology.task.ansible.out:fatal: [smithi038.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-03-19T23:06:45.590 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:45 +0000 (0:00:00.347) 0:02:47.005 ******** 2020-03-19T23:06:46.300 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-03-19T23:06:46.301 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-03-19T23:06:46.340 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.749) 0:02:47.755 ******** 2020-03-19T23:06:46.368 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-03-19T23:06:46.370 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:46.430 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.089) 0:02:47.845 ******** 2020-03-19T23:06:46.456 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-03-19T23:06:46.509 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.079) 0:02:47.924 ******** 2020-03-19T23:06:46.536 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-03-19T23:06:46.538 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:46.590 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.081) 0:02:48.005 ******** 2020-03-19T23:06:46.618 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-03-19T23:06:46.620 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:46.656 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.065) 0:02:48.071 ******** 2020-03-19T23:06:46.683 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-03-19T23:06:46.684 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:46.695 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:46.732 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.076) 0:02:48.147 ******** 2020-03-19T23:06:46.758 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-03-19T23:06:46.759 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:46.796 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.063) 0:02:48.211 ******** 2020-03-19T23:06:46.822 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-03-19T23:06:46.860 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.063) 0:02:48.275 ******** 2020-03-19T23:06:46.884 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-03-19T23:06:46.886 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:46.924 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:46 +0000 (0:00:00.064) 0:02:48.339 ******** 2020-03-19T23:06:46.952 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-03-19T23:06:46.953 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.013 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.088) 0:02:48.427 ******** 2020-03-19T23:06:47.040 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-03-19T23:06:47.041 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.077 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.064) 0:02:48.492 ******** 2020-03-19T23:06:47.103 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-03-19T23:06:47.104 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.147 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.069) 0:02:48.562 ******** 2020-03-19T23:06:47.172 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-03-19T23:06:47.173 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.208 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.061) 0:02:48.623 ******** 2020-03-19T23:06:47.234 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-03-19T23:06:47.235 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.270 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.061) 0:02:48.685 ******** 2020-03-19T23:06:47.296 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-03-19T23:06:47.297 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.345 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.075) 0:02:48.760 ******** 2020-03-19T23:06:47.371 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-03-19T23:06:47.373 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.413 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.067) 0:02:48.828 ******** 2020-03-19T23:06:47.439 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-03-19T23:06:47.440 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.479 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.066) 0:02:48.894 ******** 2020-03-19T23:06:47.505 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.545 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.065) 0:02:48.960 ******** 2020-03-19T23:06:47.571 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.610 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.064) 0:02:49.025 ******** 2020-03-19T23:06:47.635 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-03-19T23:06:47.637 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.681 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.071) 0:02:49.096 ******** 2020-03-19T23:06:47.707 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-03-19T23:06:47.708 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.746 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.065) 0:02:49.161 ******** 2020-03-19T23:06:47.783 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-03-19T23:06:47.784 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.847 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.098) 0:02:49.260 ******** 2020-03-19T23:06:47.886 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-03-19T23:06:47.886 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:47.931 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:47 +0000 (0:00:00.086) 0:02:49.346 ******** 2020-03-19T23:06:47.957 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-03-19T23:06:47.958 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.017 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.083) 0:02:49.430 ******** 2020-03-19T23:06:48.042 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-03-19T23:06:48.044 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.091 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.075) 0:02:49.506 ******** 2020-03-19T23:06:48.126 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-03-19T23:06:48.128 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.162 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.071) 0:02:49.577 ******** 2020-03-19T23:06:48.187 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-03-19T23:06:48.189 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.226 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.064) 0:02:49.641 ******** 2020-03-19T23:06:48.262 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-03-19T23:06:48.263 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.301 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.074) 0:02:49.716 ******** 2020-03-19T23:06:48.327 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-03-19T23:06:48.328 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.365 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.064) 0:02:49.780 ******** 2020-03-19T23:06:48.390 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-03-19T23:06:48.392 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.429 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.061) 0:02:49.842 ******** 2020-03-19T23:06:48.453 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T23:06:48.455 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.492 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.065) 0:02:49.907 ******** 2020-03-19T23:06:48.517 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-03-19T23:06:48.519 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.553 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.061) 0:02:49.968 ******** 2020-03-19T23:06:48.591 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-03-19T23:06:48.592 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.651 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.097) 0:02:50.066 ******** 2020-03-19T23:06:48.680 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-03-19T23:06:48.680 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.718 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.067) 0:02:50.133 ******** 2020-03-19T23:06:48.745 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.802 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.083) 0:02:50.217 ******** 2020-03-19T23:06:48.828 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-03-19T23:06:48.829 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.883 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.080) 0:02:50.298 ******** 2020-03-19T23:06:48.918 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-03-19T23:06:48.919 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:48.973 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:48 +0000 (0:00:00.090) 0:02:50.388 ******** 2020-03-19T23:06:49.003 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-03-19T23:06:49.005 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.071 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.097) 0:02:50.486 ******** 2020-03-19T23:06:49.099 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.100 INFO:teuthology.task.ansible.out:TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.142 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.070) 0:02:50.557 ******** 2020-03-19T23:06:49.143 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.171 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.172 INFO:teuthology.task.ansible.out:TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.225 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.083) 0:02:50.640 ******** 2020-03-19T23:06:49.227 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.258 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.259 INFO:teuthology.task.ansible.out:TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.306 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.080) 0:02:50.721 ******** 2020-03-19T23:06:49.307 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.346 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.347 INFO:teuthology.task.ansible.out:TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.383 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.077) 0:02:50.798 ******** 2020-03-19T23:06:49.385 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.430 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.432 INFO:teuthology.task.ansible.out:TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.468 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.084) 0:02:50.883 ******** 2020-03-19T23:06:49.469 INFO:teuthology.task.ansible.out: 2020-03-19T23:06:49.505 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-03-19T23:06:49.507 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.546 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.078) 0:02:50.961 ******** 2020-03-19T23:06:49.573 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-03-19T23:06:49.574 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.618 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.071) 0:02:51.033 ******** 2020-03-19T23:06:49.644 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-03-19T23:06:49.646 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.687 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.069) 0:02:51.102 ******** 2020-03-19T23:06:49.724 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-03-19T23:06:49.725 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.773 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.085) 0:02:51.188 ******** 2020-03-19T23:06:49.809 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-03-19T23:06:49.811 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:49.861 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:49 +0000 (0:00:00.088) 0:02:51.276 ******** 2020-03-19T23:06:50.014 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-03-19T23:06:50.016 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.169 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.308) 0:02:51.584 ******** 2020-03-19T23:06:50.194 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-03-19T23:06:50.196 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.260 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.090) 0:02:51.675 ******** 2020-03-19T23:06:50.288 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-03-19T23:06:50.289 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.328 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.068) 0:02:51.743 ******** 2020-03-19T23:06:50.364 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-03-19T23:06:50.366 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.415 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.086) 0:02:51.830 ******** 2020-03-19T23:06:50.440 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-03-19T23:06:50.442 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.477 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.062) 0:02:51.892 ******** 2020-03-19T23:06:50.502 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-03-19T23:06:50.504 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.538 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.060) 0:02:51.953 ******** 2020-03-19T23:06:50.564 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-03-19T23:06:50.565 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.609 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.071) 0:02:52.024 ******** 2020-03-19T23:06:50.638 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-03-19T23:06:50.638 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.685 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.075) 0:02:52.100 ******** 2020-03-19T23:06:50.710 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-03-19T23:06:50.712 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.747 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.062) 0:02:52.162 ******** 2020-03-19T23:06:50.772 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-03-19T23:06:50.774 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.809 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.061) 0:02:52.224 ******** 2020-03-19T23:06:50.834 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-03-19T23:06:50.837 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.871 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.061) 0:02:52.286 ******** 2020-03-19T23:06:50.898 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:50.934 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:50 +0000 (0:00:00.063) 0:02:52.349 ******** 2020-03-19T23:06:50.970 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-03-19T23:06:51.007 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:51 +0000 (0:00:00.072) 0:02:52.422 ******** 2020-03-19T23:06:51.042 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-03-19T23:06:51.043 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.064 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-03-19T23:06:51.064 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.075 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.088 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.097 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.108 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.157 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:51 +0000 (0:00:00.150) 0:02:52.572 ******** 2020-03-19T23:06:51.309 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-03-19T23:06:51.311 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003968", "end": "2020-03-19 23:06:51.291674", "rc": 0, "start": "2020-03-19 23:06:51.287706", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-03-19T23:06:51.352 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:51 +0000 (0:00:00.194) 0:02:52.767 ******** 2020-03-19T23:06:51.387 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-03-19T23:06:51.389 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.427 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:51 +0000 (0:00:00.075) 0:02:52.842 ******** 2020-03-19T23:06:51.452 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-03-19T23:06:51.454 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.492 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:51 +0000 (0:00:00.065) 0:02:52.907 ******** 2020-03-19T23:06:51.529 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-03-19T23:06:51.530 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.577 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:51 +0000 (0:00:00.084) 0:02:52.992 ******** 2020-03-19T23:06:51.602 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-03-19T23:06:51.603 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:51.652 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:51 +0000 (0:00:00.074) 0:02:53.067 ******** 2020-03-19T23:06:53.983 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-03-19T23:06:53.985 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-03-19T23:06:54.019 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:54 +0000 (0:00:02.367) 0:02:55.435 ******** 2020-03-19T23:06:54.065 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-03-19T23:06:54.065 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item={u'block_used': 12916319, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 227178789, u'size_available': 930524319744, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60987187, u'device': u'/dev/sda1', u'inode_used': 67789, u'block_size': 4096, u'options': u'rw,seclabel,relatime'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227178789, "block_size": 4096, "block_total": 240095108, "block_used": 12916319, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60987187, "inode_total": 61054976, "inode_used": 67789, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930524319744, "size_total": 983429562368, "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}} 2020-03-19T23:06:54.103 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:54 +0000 (0:00:00.083) 0:02:55.518 ******** 2020-03-19T23:06:54.335 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-03-19T23:06:54.336 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005374", "end": "2020-03-19 23:06:54.301739", "rc": 0, "start": "2020-03-19 23:06:54.296365", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:06:54.378 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:54 +0000 (0:00:00.274) 0:02:55.793 ******** 2020-03-19T23:06:54.424 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-03-19T23:06:54.426 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'block_used': 12916319, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 227178789, u'size_available': 930524319744, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60987187, u'device': u'/dev/sda1', u'inode_used': 67789, u'block_size': 4096, u'options': u'rw,seclabel,relatime'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227178789, "block_size": 4096, "block_total": 240095108, "block_used": 12916319, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60987187, "inode_total": 61054976, "inode_used": 67789, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930524319744, "size_total": 983429562368, "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:54.465 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:54 +0000 (0:00:00.087) 0:02:55.880 ******** 2020-03-19T23:06:54.491 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-03-19T23:06:54.527 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:06:54 +0000 (0:00:00.062) 0:02:55.942 ******** 2020-03-19T23:06:54.575 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-03-19T23:06:54.576 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J33D', u'wwn-0x5000c500871b5cd5'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4J33D', u'wwn': u'0x5000c500871b5cd5', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'mq-deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J33D-part1', u'wwn-0x5000c500871b5cd5-part1'], u'uuids': [u'51c8dc47-a34c-4398-9853-098c2389802a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J33D", "wwn-0x5000c500871b5cd5"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J33D-part1", "wwn-0x5000c500871b5cd5-part1"], "labels": [], "masters": [], "uuids": ["51c8dc47-a34c-4398-9853-098c2389802a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J33D", "size": "931.51 GB", "support_discard": "0", "vend 2020-03-19T23:06:54.578 INFO:teuthology.task.ansible.out:or": "ATA", "virtual": 1, "wwn": "0x5000c500871b5cd5"}}, "skip_reason": "Conditional result was False"} 2020-03-19T23:06:56.118 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-rB4COe-bY93-ipUf-2PK1-TmrK-lA2F-cTimFX', u'nvme-INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN', u'nvme-eui.0100000001000000e4d25c000034df14'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller]', u'support_discard': u'512', u'model': u'INTEL SSDPEDKE040T7', u'size': u'3.64 TB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'7814037168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.341396", "end": "2020-03-19 23:06:56.087188", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller]", "links": {"ids": ["lvm-pv-uuid-rB4COe-bY93-ipUf-2PK1-TmrK-lA2F-cTimFX", "nvme-INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN", "nvme-eui.0100000001000000e4d25c000034df14"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDKE040T7", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "7814037168", "sectorsize": "512", "size": "3.64 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:06:54.745792", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk us 2020-03-19T23:06:56.119 INFO:teuthology.task.ansible.out:ing fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T23:06:57.383 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDD36KGCGRhITse1FOYbfvNdh6nuSuiCh6'], u'uuids': [u'81295d03-157c-4027-94c9-6975588ce2d6']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.050543", "end": "2020-03-19 23:06:57.346432", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDD36KGCGRhITse1FOYbfvNdh6nuSuiCh6"], "labels": [], "masters": [], "uuids": ["81295d03-157c-4027-94c9-6975588ce2d6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:06:56.295889", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T23:06:58.660 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDh18MiP6lck47351LcIFrbazM1fxuXVN5'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.057704", "end": "2020-03-19 23:06:58.629274", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDh18MiP6lck47351LcIFrbazM1fxuXVN5"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:06:57.571570", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T23:06:59.913 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDNIdDKGfufJsVoFTufhNvCIqyAKba8qz6'], u'uuids': [u'cb25c89b-81c0-473a-a44d-84e857fd963d']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.042916", "end": "2020-03-19 23:06:59.879433", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDNIdDKGfufJsVoFTufhNvCIqyAKba8qz6"], "labels": [], "masters": [], "uuids": ["cb25c89b-81c0-473a-a44d-84e857fd963d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:06:58.836517", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T23:07:01.195 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDDXIMq1MTq28YI2blSAv1iLbjd2CIeNFL'], u'uuids': [u'0a1fc73f-1852-4ce8-9f5a-423a6c3cd3a7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.068348", "end": "2020-03-19 23:07:01.163458", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDDXIMq1MTq28YI2blSAv1iLbjd2CIeNFL"], "labels": [], "masters": [], "uuids": ["0a1fc73f-1852-4ce8-9f5a-423a6c3cd3a7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:00.095110", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T23:07:02.446 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDsa9rKSi5Pez9lf2vY0hwztoRevt8CwFJ'], u'uuids': [u'c86c7fb2-0864-4efc-9ddd-47a061be7849']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.041155", "end": "2020-03-19 23:07:02.413684", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDsa9rKSi5Pez9lf2vY0hwztoRevt8CwFJ"], "labels": [], "masters": [], "uuids": ["c86c7fb2-0864-4efc-9ddd-47a061be7849"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:01.372529", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-03-19T23:07:02.494 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:02 +0000 (0:00:07.966) 0:03:03.909 ******** 2020-03-19T23:07:02.540 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-03-19T23:07:02.541 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J33D', u'wwn-0x5000c500871b5cd5'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4J33D', u'wwn': u'0x5000c500871b5cd5', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'mq-deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4J33D-part1', u'wwn-0x5000c500871b5cd5-part1'], u'uuids': [u'51c8dc47-a34c-4398-9853-098c2389802a']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J33D", "wwn-0x5000c500871b5cd5"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J33D-part1", "wwn-0x5000c500871b5cd5-part1"], "labels": [], "masters": [], "uuids": ["51c8dc47-a34c-4398-9853-098c2389802a"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J33D", "size": "931.51 GB", "support_discard": "0", "vend 2020-03-19T23:07:02.543 INFO:teuthology.task.ansible.out:or": "ATA", "virtual": 1, "wwn": "0x5000c500871b5cd5"}}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:02.781 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-rB4COe-bY93-ipUf-2PK1-TmrK-lA2F-cTimFX', u'nvme-INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN', u'nvme-eui.0100000001000000e4d25c000034df14'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller]', u'support_discard': u'512', u'model': u'INTEL SSDPEDKE040T7', u'size': u'3.64 TB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'7814037168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_4', u'vg_nvme-lv_1', u'vg_nvme-lv_3', u'vg_nvme-lv_5', u'vg_nvme-lv_2'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036087", "end": "2020-03-19 23:07:02.757753", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_4", "vg_nvme-lv_1", "vg_nvme-lv_3", "vg_nvme-lv_5", "vg_nvme-lv_2"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller]", "links": {"ids": ["lvm-pv-uuid-rB4COe-bY93-ipUf-2PK1-TmrK-lA2F-cTimFX", "nvme-INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN", "nvme-eui.0100000001000000e4d25c000034df14"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDKE040T7", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "7814037168", "sectorsize": "512", "size": "3.64 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:02.721666", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0334023 s, 3.5 GB/s", "stderr_lines": ["110+0 records in", 2020-03-19T23:07:02.782 INFO:teuthology.task.ansible.out:"110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0334023 s, 3.5 GB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:03.171 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDD36KGCGRhITse1FOYbfvNdh6nuSuiCh6'], u'uuids': [u'81295d03-157c-4027-94c9-6975588ce2d6']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.159515", "end": "2020-03-19 23:07:03.135243", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDD36KGCGRhITse1FOYbfvNdh6nuSuiCh6"], "labels": [], "masters": [], "uuids": ["81295d03-157c-4027-94c9-6975588ce2d6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:02.975728", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.156787 s, 736 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.156787 s, 736 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:03.547 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDh18MiP6lck47351LcIFrbazM1fxuXVN5'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.174892", "end": "2020-03-19 23:07:03.519632", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDh18MiP6lck47351LcIFrbazM1fxuXVN5"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:03.344740", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.172306 s, 669 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.172306 s, 669 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:03.912 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDNIdDKGfufJsVoFTufhNvCIqyAKba8qz6'], u'uuids': [u'cb25c89b-81c0-473a-a44d-84e857fd963d']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.177423", "end": "2020-03-19 23:07:03.889969", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDNIdDKGfufJsVoFTufhNvCIqyAKba8qz6"], "labels": [], "masters": [], "uuids": ["cb25c89b-81c0-473a-a44d-84e85 2020-03-19T23:07:03.914 INFO:teuthology.task.ansible.out:7fd963d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:03.712546", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.174623 s, 661 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.174623 s, 661 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:04.255 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDDXIMq1MTq28YI2blSAv1iLbjd2CIeNFL'], u'uuids': [u'0a1fc73f-1852-4ce8-9f5a-423a6c3cd3a7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.134947", "end": "2020-03-19 23:07:04.226190", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDDXIMq1MTq28YI2blSAv1iLbjd2CIeNFL"], "labels": [], "masters": [], "uuids": ["0a1fc73f-1852-4ce8-9f5a-423a6c3cd3a7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:04.091243", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.132272 s, 872 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.132272 s, 872 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:04.594 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDsa9rKSi5Pez9lf2vY0hwztoRevt8CwFJ'], u'uuids': [u'c86c7fb2-0864-4efc-9ddd-47a061be7849']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.123510", "end": "2020-03-19 23:07:04.560284", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-zzEfOaMPVCXjt3X9ImbVr79Lkhrir0tDsa9rKSi5Pez9lf2vY0hwztoRevt8CwFJ"], "labels": [], "masters": [], "uuids": ["c86c7fb2-0864-4efc-9ddd-47a061be7849"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-03-19 23:07:04.436774", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.1208 s, 955 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.1208 s, 955 MB/s"], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:04.634 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:04 +0000 (0:00:02.139) 0:03:06.049 ******** 2020-03-19T23:07:04.907 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-03-19T23:07:04.909 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.065677", "end": "2020-03-19 23:07:04.883010", "rc": 0, "start": "2020-03-19 23:07:04.817333", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:04.946 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:04 +0000 (0:00:00.311) 0:03:06.360 ******** 2020-03-19T23:07:05.230 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-03-19T23:07:05.232 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.072126", "end": "2020-03-19 23:07:05.202776", "rc": 0, "start": "2020-03-19 23:07:05.130650", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:05.267 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.322) 0:03:06.682 ******** 2020-03-19T23:07:05.294 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-03-19T23:07:05.333 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.065) 0:03:06.748 ******** 2020-03-19T23:07:05.369 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-03-19T23:07:05.371 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:05.410 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.077) 0:03:06.825 ******** 2020-03-19T23:07:05.436 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-03-19T23:07:05.438 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:05.475 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.064) 0:03:06.890 ******** 2020-03-19T23:07:05.501 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-03-19T23:07:05.502 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:05.541 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.066) 0:03:06.956 ******** 2020-03-19T23:07:05.568 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-03-19T23:07:05.569 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:05.604 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.063) 0:03:07.019 ******** 2020-03-19T23:07:05.828 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-03-19T23:07:05.830 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-03-19T23:07:05.866 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.262) 0:03:07.281 ******** 2020-03-19T23:07:05.903 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-03-19T23:07:05.903 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={u'block_used': 12916319, u'uuid': u'51c8dc47-a34c-4398-9853-098c2389802a', u'size_total': 983429562368, u'block_total': 240095108, u'mount': u'/', u'block_available': 227178789, u'size_available': 930524319744, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60987187, u'device': u'/dev/sda1', u'inode_used': 67789, u'block_size': 4096, u'options': u'rw,seclabel,relatime'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227178789, "block_size": 4096, "block_total": 240095108, "block_used": 12916319, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60987187, "inode_total": 61054976, "inode_used": 67789, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930524319744, "size_total": 983429562368, "uuid": "51c8dc47-a34c-4398-9853-098c2389802a"}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:05.964 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:05 +0000 (0:00:00.097) 0:03:07.379 ******** 2020-03-19T23:07:06.000 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-03-19T23:07:06.002 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:06.038 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:06 +0000 (0:00:00.073) 0:03:07.453 ******** 2020-03-19T23:07:06.077 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:06.115 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:06 +0000 (0:00:00.077) 0:03:07.530 ******** 2020-03-19T23:07:06.140 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T23:07:06.142 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:06.182 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:06 +0000 (0:00:00.066) 0:03:07.597 ******** 2020-03-19T23:07:06.207 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-03-19T23:07:06.209 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:06.248 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:06 +0000 (0:00:00.066) 0:03:07.663 ******** 2020-03-19T23:07:06.284 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-03-19T23:07:06.285 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:06.326 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:06 +0000 (0:00:00.077) 0:03:07.741 ******** 2020-03-19T23:07:06.353 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-03-19T23:07:06.355 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:06.401 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:06 +0000 (0:00:00.074) 0:03:07.816 ******** 2020-03-19T23:07:07.076 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-03-19T23:07:07.077 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-03-19T23:07:07.129 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:07 +0000 (0:00:00.728) 0:03:08.544 ******** 2020-03-19T23:07:07.698 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-03-19T23:07:07.699 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T23:07:08.052 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T23:07:08.415 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T23:07:08.775 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T23:07:09.149 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-03-19T23:07:09.209 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:09 +0000 (0:00:02.080) 0:03:10.624 ******** 2020-03-19T23:07:09.423 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-03-19T23:07:09.424 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-03-19T23:07:09.489 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:09 +0000 (0:00:00.279) 0:03:10.904 ******** 2020-03-19T23:07:09.550 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-03-19T23:07:09.551 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:09.747 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T23:07:09.934 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T23:07:10.143 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T23:07:10.341 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-03-19T23:07:10.385 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:10 +0000 (0:00:00.895) 0:03:11.800 ******** 2020-03-19T23:07:10.600 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-03-19T23:07:10.601 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-03-19T23:07:10.663 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:10 +0000 (0:00:00.278) 0:03:12.078 ******** 2020-03-19T23:07:10.700 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-03-19T23:07:10.702 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:10.755 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:10 +0000 (0:00:00.092) 0:03:12.170 ******** 2020-03-19T23:07:11.389 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-03-19T23:07:11.390 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true} 2020-03-19T23:07:11.428 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:11 +0000 (0:00:00.672) 0:03:12.843 ******** 2020-03-19T23:07:11.899 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2020-03-19T23:07:11.900 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-03-19T23:07:11.937 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:11 +0000 (0:00:00.508) 0:03:13.352 ******** 2020-03-19T23:07:11.962 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-03-19T23:07:11.964 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:11.999 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:11 +0000 (0:00:00.062) 0:03:13.414 ******** 2020-03-19T23:07:12.025 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-03-19T23:07:12.027 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.062 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.062) 0:03:13.477 ******** 2020-03-19T23:07:12.088 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-03-19T23:07:12.089 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.135 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.072) 0:03:13.550 ******** 2020-03-19T23:07:12.160 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-03-19T23:07:12.161 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.200 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.064) 0:03:13.615 ******** 2020-03-19T23:07:12.225 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-03-19T23:07:12.227 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.263 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.063) 0:03:13.678 ******** 2020-03-19T23:07:12.289 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-03-19T23:07:12.290 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.328 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.064) 0:03:13.743 ******** 2020-03-19T23:07:12.353 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-03-19T23:07:12.355 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.395 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.067) 0:03:13.810 ******** 2020-03-19T23:07:12.420 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-03-19T23:07:12.421 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.457 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.062) 0:03:13.872 ******** 2020-03-19T23:07:12.485 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-03-19T23:07:12.487 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.523 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.065) 0:03:13.938 ******** 2020-03-19T23:07:12.549 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-03-19T23:07:12.550 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.590 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.067) 0:03:14.005 ******** 2020-03-19T23:07:12.615 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-03-19T23:07:12.617 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:12.652 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:12 +0000 (0:00:00.061) 0:03:14.067 ******** 2020-03-19T23:07:13.162 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-03-19T23:07:13.162 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-03-19T23:07:13.201 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:13 +0000 (0:00:00.549) 0:03:14.616 ******** 2020-03-19T23:07:13.526 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-03-19T23:07:13.528 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2020-03-19 23:03:09 UTC", "ActiveEnterTimestampMonotonic": "14290502", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket -.mount sysinit.target basic.target tmp.mount sntp.service ntpdate.service systemd-tmpfiles-setup.service system.slice ntpd.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Thu 2020-03-19 23:03:08 UTC", "AssertTimestampMonotonic": "13087449", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2020-03-19 23:03:08 UTC", "ConditionTimestampMonotonic": "13087422", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-03-19T23:07:13.529 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2335", "ExecMainStartTimestamp": "Thu 2020-03-19 23:03:09 UTC", "ExecMainStartTimestampMonotonic": "14261929", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Thu 2020-03-19 23:03:08 UTC", "InactiveExitTimestampMonotonic": "13118550", "InvocationID": "25dd2d01b6c54206a38b2c8cec0875f4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-03-19T23:07:13.530 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127467", "LimitNPROCSoft": "127467", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127467", "LimitSIGPENDINGSoft": "127467", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2335", "MemoryAccounting": "yes", "MemoryCurrent": "2744320", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-03-19T23:07:13.532 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2020-03-19 23:03:09 UTC", "StateChangeTimestampMonotonic": "14290502", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Thu 2020-03-19 23:03:09 UTC", "WatchdogTimestampMonotonic": "14261943", "WatchdogUSec": "0"}} 2020-03-19T23:07:13.564 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:13 +0000 (0:00:00.363) 0:03:14.979 ******** 2020-03-19T23:07:13.589 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-03-19T23:07:13.591 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:13.625 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:13 +0000 (0:00:00.060) 0:03:15.040 ******** 2020-03-19T23:07:13.649 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-03-19T23:07:13.651 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:13.685 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:13 +0000 (0:00:00.060) 0:03:15.100 ******** 2020-03-19T23:07:13.720 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-03-19T23:07:13.722 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:13.762 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:13 +0000 (0:00:00.076) 0:03:15.177 ******** 2020-03-19T23:07:13.983 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-03-19T23:07:13.983 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-03-19T23:07:14.030 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:14 +0000 (0:00:00.267) 0:03:15.445 ******** 2020-03-19T23:07:14.540 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-03-19T23:07:14.540 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1211, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-03-19T23:07:14.577 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:14 +0000 (0:00:00.546) 0:03:15.992 ******** 2020-03-19T23:07:15.791 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-03-19T23:07:15.792 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-03-19T23:07:15.830 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:15 +0000 (0:00:01.253) 0:03:17.245 ******** 2020-03-19T23:07:15.855 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-03-19T23:07:15.856 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:15.897 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:15 +0000 (0:00:00.067) 0:03:17.312 ******** 2020-03-19T23:07:15.924 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-03-19T23:07:15.925 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-03-19T23:07:15.961 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:15 +0000 (0:00:00.063) 0:03:17.376 ******** 2020-03-19T23:07:16.171 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-03-19T23:07:16.172 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-03-19T23:07:16.206 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:16 +0000 (0:00:00.245) 0:03:17.621 ******** 2020-03-19T23:07:16.231 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-03-19T23:07:16.232 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:16.267 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:16 +0000 (0:00:00.061) 0:03:17.682 ******** 2020-03-19T23:07:16.293 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-03-19T23:07:16.295 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-03-19T23:07:16.329 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:16 +0000 (0:00:00.061) 0:03:17.744 ******** 2020-03-19T23:07:16.354 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-03-19T23:07:16.355 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:16.390 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:16 +0000 (0:00:00.061) 0:03:17.805 ******** 2020-03-19T23:07:16.591 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-03-19T23:07:16.615 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-03-19T23:07:16.649 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:16 +0000 (0:00:00.259) 0:03:18.064 ******** 2020-03-19T23:07:16.862 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-03-19T23:07:16.864 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-03-19T23:07:16.900 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:16 +0000 (0:00:00.251) 0:03:18.315 ******** 2020-03-19T23:07:16.925 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-03-19T23:07:16.927 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:16.965 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:16 +0000 (0:00:00.064) 0:03:18.380 ******** 2020-03-19T23:07:17.002 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-03-19T23:07:17.003 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi038.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.014 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.015 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.053 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.087) 0:03:18.468 ******** 2020-03-19T23:07:17.090 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-03-19T23:07:17.091 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.101 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.112 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.114 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.124 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.163 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.110) 0:03:18.578 ******** 2020-03-19T23:07:17.191 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-03-19T23:07:17.193 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.242 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.078) 0:03:18.657 ******** 2020-03-19T23:07:17.267 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-03-19T23:07:17.269 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.318 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.075) 0:03:18.733 ******** 2020-03-19T23:07:17.345 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-03-19T23:07:17.346 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.387 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.069) 0:03:18.802 ******** 2020-03-19T23:07:17.415 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-03-19T23:07:17.416 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.457 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.069) 0:03:18.872 ******** 2020-03-19T23:07:17.492 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-03-19T23:07:17.493 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.530 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.073) 0:03:18.945 ******** 2020-03-19T23:07:17.556 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-03-19T23:07:17.557 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.594 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.063) 0:03:19.009 ******** 2020-03-19T23:07:17.619 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-03-19T23:07:17.620 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.657 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.063) 0:03:19.072 ******** 2020-03-19T23:07:17.692 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-03-19T23:07:17.697 INFO:teuthology.task.ansible.out:skipping: [smithi038.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-03-19T23:07:17.750 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:17 +0000 (0:00:00.092) 0:03:19.165 ******** 2020-03-19T23:07:18.996 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-03-19T23:07:18.997 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:01.006793", "end": "2020-03-19 23:07:18.959992", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-03-19 23:07:17.953199", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:19.203 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003134", "end": "2020-03-19 23:07:19.173174", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-03-19 23:07:19.170040", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:19.427 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002810", "end": "2020-03-19 23:07:19.366565", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-03-19 23:07:19.363755", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:19.648 INFO:teuthology.task.ansible.out:ok: [smithi038.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.020517", "end": "2020-03-19 23:07:19.608679", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-03-19 23:07:19.588162", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-03-19T23:07:19.701 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:19 +0000 (0:00:01.951) 0:03:21.116 ******** 2020-03-19T23:07:19.727 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-03-19T23:07:19.776 INFO:teuthology.task.ansible.out:Thursday 19 March 2020 23:07:19 +0000 (0:00:00.074) 0:03:21.191 ******** 2020-03-19T23:07:19.989 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-03-19T23:07:19.990 INFO:teuthology.task.ansible.out:changed: [smithi038.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-03-19T23:07:20.122 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** 2020-03-19T23:07:20.123 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-03-19T23:07:20.127 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2020-03-19T23:07:20.127 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-03-19T23:07:20.131 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** 2020-03-19T23:07:20.132 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2020-03-19T23:07:20.133 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi038.front.sepia.ceph.com : ok=121 changed=35 unreachable=0 failed=0 skipped=162 rescued=0 ignored=4 Thursday 19 March 2020 23:07:20 +0000 (0:00:00.356) 0:03:21.547 ******** =============================================================================== 2020-03-19T23:07:20.135 INFO:teuthology.task.ansible.out:common : nrpe - Remove SELinux policy package -------------------------- 17.15s common : nrpe - Load SELinux policy package ---------------------------- 17.07s testnode : Install packages -------------------------------------------- 11.92s testnode : Base group workaround for BZ1782899 ------------------------- 11.41s common : Enable necessary rhsm repos ----------------------------------- 11.14s common : Get list of enabled RHSM repos -------------------------------- 10.50s common : Register with subscription-manager. ---------------------------- 9.45s 2020-03-19T23:07:20.136 INFO:teuthology.task.ansible.out:testnode : Zap all non-root disks --------------------------------------- 7.97s testnode : Ensure ceph dependency packages are not present. ------------- 6.66s testnode : Ensure ceph packages are not present. ------------------------ 6.09s common : Disable all rhsm repos ----------------------------------------- 5.82s Gathering Facts --------------------------------------------------------- 2.59s common : Install Kerberos Packages (RedHat) ----------------------------- 2.45s testnode : Make sure rpm dependencies are installed --------------------- 2.37s testnode : Install epel packages ---------------------------------------- 2.31s common : nrpe - Ensure SELinux policy is up to date --------------------- 2.16s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.14s common : nrpe - Install SELinux dependencies ---------------------------- 2.13s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 2.12s testnode : Remove packages ---------------------------------------------- 2.09s 2020-03-19T23:07:20.415 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com')] 2020-03-19T23:07:20.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-19T23:07:20.417 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi038.front.sepia.ceph.com', 'timeout': 60} 2020-03-19T23:07:20.489 INFO:teuthology.orchestra.run.smithi038:> true 2020-03-19T23:07:20.574 INFO:teuthology.run_tasks:Running task clock... 2020-03-19T23:07:20.586 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-03-19T23:07:20.586 INFO:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-03-19T23:07:20.652 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-03-19T23:07:20.674 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-03-19T23:07:20.726 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2020-03-19T23:07:20.743 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2020-03-19T23:07:20.766 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-03-19T23:07:20.785 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-03-19T23:07:20.853 INFO:teuthology.orchestra.run.smithi038.stderr:bash: ntpq: command not found 2020-03-19T23:07:20.856 INFO:teuthology.orchestra.run.smithi038.stdout:210 Number of sources = 4 2020-03-19T23:07:20.856 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-03-19T23:07:20.856 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2020-03-19T23:07:20.857 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T23:07:20.857 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T23:07:20.857 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T23:07:20.857 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T23:07:20.858 INFO:teuthology.run_tasks:Running task install... 2020-03-19T23:07:21.168 DEBUG:teuthology.task.install:project ceph 2020-03-19T23:07:21.169 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}} 2020-03-19T23:07:21.169 DEBUG:teuthology.task.install:config {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'} 2020-03-19T23:07:21.169 INFO:teuthology.task.install:Using flavor: basic 2020-03-19T23:07:21.177 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-03-19T23:07:21.177 INFO:teuthology.task.install:extra packages: [] 2020-03-19T23:07:21.177 INFO:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2020-03-19T23:07:21.219 INFO:teuthology.orchestra.run.smithi038.stdout:RedHatEnterprise 2020-03-19T23:07:21.220 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T23:07:21.221 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'basic', 'extras': None} 2020-03-19T23:07:21.221 INFO:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2020-03-19T23:07:21.301 INFO:teuthology.orchestra.run.smithi038.stdout:RedHatEnterprise 2020-03-19T23:07:21.302 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T23:07:21.302 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=787db24c312ed9d310cd3f194b00548ede9a199c 2020-03-19T23:07:21.513 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/ 2020-03-19T23:07:21.514 INFO:teuthology.task.install.rpm:Package version is 15.1.1-234.g787db24 2020-03-19T23:07:21.738 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-sage-testing-2020-03-19-0855/787db24c312ed9d310cd3f194b00548ede9a199c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-03-19T23:07:21.739 INFO:teuthology.orchestra.run.smithi038:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-03-19T23:07:21.790 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2020-03-19T23:07:21.790 INFO:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/787db24c312ed9d310cd3f194b00548ede9a199c/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-03-19T23:07:21.863 INFO:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-03-19T23:07:21.952 INFO:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-03-19T23:07:21.983 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2020-03-19T23:07:21.984 INFO:teuthology.orchestra.run.smithi038:> sudo yum clean all 2020-03-19T23:07:22.375 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:07:23.137 INFO:teuthology.orchestra.run.smithi038.stdout:44 files removed 2020-03-19T23:07:23.183 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw 2020-03-19T23:07:23.533 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:07:24.542 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 179 kB/s | 69 kB 00:00 2020-03-19T23:07:24.896 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 42 kB/s | 14 kB 00:00 2020-03-19T23:07:25.193 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 5.1 kB/s | 1.4 kB 00:00 2020-03-19T23:07:25.311 INFO:teuthology.orchestra.run.smithi038.stdout:Copr repo for ceph-el8 owned by ktdreyer 386 kB/s | 38 kB 00:00 2020-03-19T23:07:25.610 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 22 MB/s | 6.1 MB 00:00 2020-03-19T23:07:26.861 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 5.2 MB/s | 30 kB 00:00 2020-03-19T23:07:27.141 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 14 MB 00:00 2020-03-19T23:07:29.523 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 14 MB 00:00 2020-03-19T23:07:33.231 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:07:33.234 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:07:33.235 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2020-03-19T23:07:33.235 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:07:33.235 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:07:33.235 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:15.1.1-234.g787db24.el8 ceph 9.8 M 2020-03-19T23:07:33.235 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2020-03-19T23:07:33.235 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:15.1.1-234.g787db24.el8 ceph 3.6 M 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:15.1.1-234.g787db24.el8 ceph 1.9 M 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:15.1.1-234.g787db24.el8 ceph 5.5 M 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:15.1.1-234.g787db24.el8 ceph 21 M 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:15.1.1-234.g787db24.el8 ceph 25 k 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:15.1.1-234.g787db24.el8 ceph 508 k 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:15.1.1-234.g787db24.el8 ceph 377 k 2020-03-19T23:07:33.236 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:15.1.1-234.g787db24.el8 ceph 6.2 M 2020-03-19T23:07:33.237 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:15.1.1-234.g787db24.el8 ceph 40 k 2020-03-19T23:07:33.237 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:15.1.1-234.g787db24.el8 ceph 60 k 2020-03-19T23:07:33.237 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:15.1.1-234.g787db24.el8 ceph 162 k 2020-03-19T23:07:33.237 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:15.1.1-234.g787db24.el8 ceph 352 k 2020-03-19T23:07:33.237 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:15.1.1-234.g787db24.el8 ceph 309 k 2020-03-19T23:07:33.237 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:15.1.1-234.g787db24.el8 ceph 113 k 2020-03-19T23:07:33.237 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.7-6.el8 epel 315 k 2020-03-19T23:07:33.238 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2020-03-19T23:07:33.238 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2020-03-19T23:07:33.238 INFO:teuthology.orchestra.run.smithi038.stdout: leveldb x86_64 1.21-2.el8 lab-extras 179 k 2020-03-19T23:07:33.238 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.8.1-11.el8 lab-extras 257 k 2020-03-19T23:07:33.238 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 2020-03-19T23:07:33.238 INFO:teuthology.orchestra.run.smithi038.stdout: 44 k 2020-03-19T23:07:33.238 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout: 353 k 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.9.0-1.el8 rhel-8-for-x86_64-baseos-rpms 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout: 47 k 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout:Install 21 Packages 2020-03-19T23:07:33.239 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 2 Packages 2020-03-19T23:07:33.240 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:07:33.240 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 51 M 2020-03-19T23:07:33.240 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:07:35.098 INFO:teuthology.orchestra.run.smithi038.stdout:(1/23): ceph-base-15.1.1-234.g787db24.el8.x86_6 2.9 MB/s | 5.5 MB 00:01 2020-03-19T23:07:35.175 INFO:teuthology.orchestra.run.smithi038.stdout:(2/23): ceph-selinux-15.1.1-234.g787db24.el8.x8 321 kB/s | 25 kB 00:00 2020-03-19T23:07:35.347 INFO:teuthology.orchestra.run.smithi038.stdout:(3/23): libcephfs2-15.1.1-234.g787db24.el8.x86_ 2.9 MB/s | 508 kB 00:00 2020-03-19T23:07:35.517 INFO:teuthology.orchestra.run.smithi038.stdout:(4/23): ceph-common-15.1.1-234.g787db24.el8.x86 9.1 MB/s | 21 MB 00:02 2020-03-19T23:07:35.521 INFO:teuthology.orchestra.run.smithi038.stdout:(5/23): libradosstriper1-15.1.1-234.g787db24.el 2.1 MB/s | 377 kB 00:00 2020-03-19T23:07:35.611 INFO:teuthology.orchestra.run.smithi038.stdout:(6/23): python3-ceph-argparse-15.1.1-234.g787db 444 kB/s | 40 kB 00:00 2020-03-19T23:07:35.695 INFO:teuthology.orchestra.run.smithi038.stdout:(7/23): python3-ceph-common-15.1.1-234.g787db24 713 kB/s | 60 kB 00:00 2020-03-19T23:07:35.787 INFO:teuthology.orchestra.run.smithi038.stdout:(8/23): python3-cephfs-15.1.1-234.g787db24.el8. 1.7 MB/s | 162 kB 00:00 2020-03-19T23:07:35.962 INFO:teuthology.orchestra.run.smithi038.stdout:(9/23): python3-rados-15.1.1-234.g787db24.el8.x 2.0 MB/s | 352 kB 00:00 2020-03-19T23:07:36.119 INFO:teuthology.orchestra.run.smithi038.stdout:(10/23): librgw2-15.1.1-234.g787db24.el8.x86_64 10 MB/s | 6.2 MB 00:00 2020-03-19T23:07:36.122 INFO:teuthology.orchestra.run.smithi038.stdout:(11/23): python3-rbd-15.1.1-234.g787db24.el8.x8 1.9 MB/s | 309 kB 00:00 2020-03-19T23:07:36.170 INFO:teuthology.orchestra.run.smithi038.stdout:(12/23): python3-rgw-15.1.1-234.g787db24.el8.x8 2.2 MB/s | 113 kB 00:00 2020-03-19T23:07:36.237 INFO:teuthology.orchestra.run.smithi038.stdout:(13/23): gperftools-libs-2.7-6.el8.x86_64.rpm 2.7 MB/s | 315 kB 00:00 2020-03-19T23:07:36.245 INFO:teuthology.orchestra.run.smithi038.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 781 kB/s | 59 kB 00:00 2020-03-19T23:07:36.254 INFO:teuthology.orchestra.run.smithi038.stdout:(15/23): leveldb-1.21-2.el8.x86_64.rpm 21 MB/s | 179 kB 00:00 2020-03-19T23:07:36.256 INFO:teuthology.orchestra.run.smithi038.stdout:(16/23): libunwind-1.3.1-3.el8.x86_64.rpm 3.8 MB/s | 75 kB 00:00 2020-03-19T23:07:36.258 INFO:teuthology.orchestra.run.smithi038.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 66 MB/s | 257 kB 00:00 2020-03-19T23:07:36.314 INFO:teuthology.orchestra.run.smithi038.stdout:(18/23): librdkafka-0.11.4-1.el8.x86_64.rpm 6.2 MB/s | 353 kB 00:00 2020-03-19T23:07:36.318 INFO:teuthology.orchestra.run.smithi038.stdout:(19/23): python3-prettytable-0.7.2-14.el8.noarc 716 kB/s | 44 kB 00:00 2020-03-19T23:07:36.343 INFO:teuthology.orchestra.run.smithi038.stdout:(20/23): librabbitmq-0.9.0-1.el8.x86_64.rpm 1.6 MB/s | 47 kB 00:00 2020-03-19T23:07:36.581 INFO:teuthology.orchestra.run.smithi038.stdout:(21/23): librbd1-15.1.1-234.g787db24.el8.x86_64 8.0 MB/s | 1.9 MB 00:00 2020-03-19T23:07:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:(22/23): librados2-15.1.1-234.g787db24.el8.x86_ 4.8 MB/s | 3.6 MB 00:00 2020-03-19T23:07:37.447 INFO:teuthology.orchestra.run.smithi038.stdout:(23/23): ceph-radosgw-15.1.1-234.g787db24.el8.x 2.3 MB/s | 9.8 MB 00:04 2020-03-19T23:07:37.451 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:07:37.451 INFO:teuthology.orchestra.run.smithi038.stdout:Total 12 MB/s | 51 MB 00:04 2020-03-19T23:07:37.457 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:07:37.506 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:07:37.506 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:07:37.863 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:07:37.864 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:07:38.943 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:07:39.109 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2020-03-19T23:07:39.535 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2020-03-19T23:07:39.692 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:15.1.1-234.g787db24.el8.x86_64 2/25 2020-03-19T23:07:39.800 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:15.1.1-234.g787db24.el8.x86_64 2/25 2020-03-19T23:07:39.998 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 3/25 2020-03-19T23:07:40.254 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2020-03-19T23:07:40.492 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 5/25 2020-03-19T23:07:40.724 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 5/25 2020-03-19T23:07:40.909 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:15.1.1-234.g787db24.el8.x86_64 6/25 2020-03-19T23:07:40.958 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:15.1.1-234.g787db24.el8.x86_64 6/25 2020-03-19T23:07:41.161 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 7/25 2020-03-19T23:07:41.284 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/25 2020-03-19T23:07:41.952 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/25 2020-03-19T23:07:42.167 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:15.1.1-234.g787db24.el8.x86_64 9/25 2020-03-19T23:07:42.261 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:15.1.1-234.g787db24.el8.x86_64 9/25 2020-03-19T23:07:42.378 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 10/25 2020-03-19T23:07:42.579 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 11/25 2020-03-19T23:07:42.773 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 12/25 2020-03-19T23:07:43.011 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 13/25 2020-03-19T23:07:43.192 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 14/25 2020-03-19T23:07:43.267 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 14/25 2020-03-19T23:07:43.434 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 15/25 2020-03-19T23:07:43.683 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2020-03-19T23:07:43.926 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 17/25 2020-03-19T23:07:44.135 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : leveldb-1.21-2.el8.x86_64 18/25 2020-03-19T23:07:44.284 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 19/25 2020-03-19T23:07:47.938 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 20/25 2020-03-19T23:07:48.175 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 20/25 2020-03-19T23:07:48.909 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 20/25 2020-03-19T23:07:49.165 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 21/25 2020-03-19T23:07:49.251 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 21/25 2020-03-19T23:07:49.366 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 22/25 2020-03-19T23:08:06.169 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 22/25 2020-03-19T23:08:06.169 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2020-03-19T23:08:06.169 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2020-03-19T23:08:06.169 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2020-03-19T23:08:06.169 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /run 2020-03-19T23:08:06.169 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2020-03-19T23:08:06.170 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2020-03-19T23:08:06.170 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2020-03-19T23:08:06.170 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2020-03-19T23:08:06.170 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2020-03-19T23:08:06.170 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:07.272 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 23/25 2020-03-19T23:08:07.377 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 23/25 2020-03-19T23:08:07.377 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2020-03-19T23:08:07.558 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2020-03-19T23:08:07.558 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2020-03-19T23:08:08.259 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2020-03-19T23:08:08.259 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 1/25 2020-03-19T23:08:08.259 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2/25 2020-03-19T23:08:08.259 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 3/25 2020-03-19T23:08:08.259 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 4/25 2020-03-19T23:08:08.259 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 5/25 2020-03-19T23:08:08.259 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 6/25 2020-03-19T23:08:08.260 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:15.1.1-234.g787db24.el8.x86_64 7/25 2020-03-19T23:08:08.260 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 8/25 2020-03-19T23:08:08.260 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 9/25 2020-03-19T23:08:08.260 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 10/25 2020-03-19T23:08:08.260 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 11/25 2020-03-19T23:08:08.260 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 12/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 13/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 14/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : leveldb-1.21-2.el8.x86_64 17/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 18/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2020-03-19T23:08:08.261 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/25 2020-03-19T23:08:08.262 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 21/25 2020-03-19T23:08:08.262 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:15.1.1-234.g787db24.el8.x86_64 22/25 2020-03-19T23:08:08.262 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2020-03-19T23:08:08.262 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:15.1.1-234.g787db24.el8.x86_64 24/25 2020-03-19T23:08:11.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2020-03-19T23:08:11.545 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:08:11.736 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:11.737 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2020-03-19T23:08:11.737 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.737 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.737 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:11.737 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:08:11.737 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-03-19T23:08:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.2-3.el8.x86_64 2020-03-19T23:08:11.740 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.3.1-3.el8.x86_64 2020-03-19T23:08:11.740 INFO:teuthology.orchestra.run.smithi038.stdout: leveldb-1.21-2.el8.x86_64 2020-03-19T23:08:11.740 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-03-19T23:08:11.740 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-03-19T23:08:11.740 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-03-19T23:08:11.740 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-03-19T23:08:11.740 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:11.741 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:08:11.852 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-test 2020-03-19T23:08:12.215 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:08:13.283 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 66 kB/s | 2.8 kB 00:00 2020-03-19T23:08:13.378 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 kB/s | 2.4 kB 00:00 2020-03-19T23:08:14.157 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:08:14.158 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:14.158 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2020-03-19T23:08:14.158 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:14.158 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:08:14.159 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:15.1.1-234.g787db24.el8 2020-03-19T23:08:14.159 INFO:teuthology.orchestra.run.smithi038.stdout: ceph 42 M 2020-03-19T23:08:14.159 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:08:14.159 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2020-03-19T23:08:14.160 INFO:teuthology.orchestra.run.smithi038.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2020-03-19T23:08:14.160 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.3.2-6.el8 rhel-8-for-x86_64-appstream-rpms 298 k 2020-03-19T23:08:14.160 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma x86_64 6.8.2-1.el8 rhel-8-for-x86_64-appstream-rpms 188 k 2020-03-19T23:08:14.160 INFO:teuthology.orchestra.run.smithi038.stdout: libxslt x86_64 1.1.32-3.el8 rhel-8-for-x86_64-baseos-rpms 249 k 2020-03-19T23:08:14.161 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:14.161 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:08:14.161 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:14.161 INFO:teuthology.orchestra.run.smithi038.stdout:Install 6 Packages 2020-03-19T23:08:14.161 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:14.162 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 43 M 2020-03-19T23:08:14.163 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 185 M 2020-03-19T23:08:14.163 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:08:14.202 INFO:teuthology.orchestra.run.smithi038.stdout:(1/6): jq-1.5-12.el8.x86_64.rpm 3.8 MB/s | 161 kB 00:00 2020-03-19T23:08:14.225 INFO:teuthology.orchestra.run.smithi038.stdout:(2/6): xmlstarlet-1.6.1-11.el8.x86_64.rpm 1.1 MB/s | 69 kB 00:00 2020-03-19T23:08:14.236 INFO:teuthology.orchestra.run.smithi038.stdout:(3/6): socat-1.7.3.2-6.el8.x86_64.rpm 8.9 MB/s | 298 kB 00:00 2020-03-19T23:08:14.270 INFO:teuthology.orchestra.run.smithi038.stdout:(4/6): oniguruma-6.8.2-1.el8.x86_64.rpm 3.8 MB/s | 188 kB 00:00 2020-03-19T23:08:14.276 INFO:teuthology.orchestra.run.smithi038.stdout:(5/6): libxslt-1.1.32-3.el8.x86_64.rpm 6.0 MB/s | 249 kB 00:00 2020-03-19T23:08:16.275 INFO:teuthology.orchestra.run.smithi038.stdout:(6/6): ceph-test-15.1.1-234.g787db24.el8.x86_64 20 MB/s | 42 MB 00:02 2020-03-19T23:08:16.276 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:08:16.277 INFO:teuthology.orchestra.run.smithi038.stdout:Total 20 MB/s | 43 MB 00:02 2020-03-19T23:08:16.284 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:08:16.306 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:08:16.307 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:08:16.594 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:08:16.594 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:08:17.314 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:08:17.519 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libxslt-1.1.32-3.el8.x86_64 1/6 2020-03-19T23:08:17.675 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-03-19T23:08:17.845 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 3/6 2020-03-19T23:08:17.923 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 3/6 2020-03-19T23:08:18.141 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : jq-1.5-12.el8.x86_64 4/6 2020-03-19T23:08:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.3.2-6.el8.x86_64 5/6 2020-03-19T23:08:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 6/6 2020-03-19T23:08:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-test-2:15.1.1-234.g787db24.el8.x86_64 6/6 2020-03-19T23:08:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 1/6 2020-03-19T23:08:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-03-19T23:08:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : jq-1.5-12.el8.x86_64 3/6 2020-03-19T23:08:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 4/6 2020-03-19T23:08:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 5/6 2020-03-19T23:08:26.800 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 6/6 2020-03-19T23:08:26.801 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:08:26.992 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:26.993 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:08:26.993 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:15.1.1-234.g787db24.el8.x86_64 xmlstarlet-1.6.1-11.el8.x86_64 2020-03-19T23:08:26.993 INFO:teuthology.orchestra.run.smithi038.stdout: jq-1.5-12.el8.x86_64 socat-1.7.3.2-6.el8.x86_64 2020-03-19T23:08:26.993 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma-6.8.2-1.el8.x86_64 libxslt-1.1.32-3.el8.x86_64 2020-03-19T23:08:26.993 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:26.993 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:08:27.187 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph 2020-03-19T23:08:27.542 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:08:29.099 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 kB/s | 2.8 kB 00:00 2020-03-19T23:08:29.204 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 kB/s | 2.4 kB 00:00 2020-03-19T23:08:30.028 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:08:30.029 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:30.029 INFO:teuthology.orchestra.run.smithi038.stdout: Package ArchVersion Repository Size 2020-03-19T23:08:30.029 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:30.029 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:08:30.029 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_642:15.1.1-234.g787db24.el8 ceph 6.4 k 2020-03-19T23:08:30.030 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:08:30.030 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_642:15.1.1-234.g787db24.el8 ceph 2.0 M 2020-03-19T23:08:30.030 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_642:15.1.1-234.g787db24.el8 ceph 1.4 M 2020-03-19T23:08:30.030 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_642:15.1.1-234.g787db24.el8 ceph 4.4 M 2020-03-19T23:08:30.030 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_642:15.1.1-234.g787db24.el8 ceph 17 M 2020-03-19T23:08:30.030 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch2:15.1.1-234.g787db24.el8 ceph-noarch 163 k 2020-03-19T23:08:30.030 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch8.2.1-1.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 145 k 2020-03-19T23:08:30.031 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch18.4.0-1.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 385 k 2020-03-19T23:08:30.031 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch6.2-6.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 11 k 2020-03-19T23:08:30.031 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch2.0-4.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 18 k 2020-03-19T23:08:30.031 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch0.3.5-11.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 49 k 2020-03-19T23:08:30.031 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch7.2.0-3.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 59 k 2020-03-19T23:08:30.031 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch1.3.2-9.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 283 k 2020-03-19T23:08:30.031 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch2.6-1.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 16 k 2020-03-19T23:08:30.032 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric noarch0.8.1-17.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 19 k 2020-03-19T23:08:30.032 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch noarch3.4.0.3-18.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 24 k 2020-03-19T23:08:30.032 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch1.14.1-5.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 30 k 2020-03-19T23:08:30.032 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme noarch0.5.2-4.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 27 k 2020-03-19T23:08:30.032 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch2.0-2.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 23 k 2020-03-19T23:08:30.032 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_643.1.6-2.el8.1 epel 44 k 2020-03-19T23:08:30.032 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 epel 185 k 2020-03-19T23:08:30.033 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress noarch1.2.1-2.el8.1 epel 241 k 2020-03-19T23:08:30.033 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch1.8.5-1.el8.1 epel 251 k 2020-03-19T23:08:30.033 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest noarch2.0.33-1.el8 epel 85 k 2020-03-19T23:08:30.033 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webencodings noarch0.5.1-6.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-03-19T23:08:30.033 INFO:teuthology.orchestra.run.smithi038.stdout: python3-lxml x86_644.2.3-1.el8 rhel-8-for-x86_64-appstream-rpms 1.5 M 2020-03-19T23:08:30.034 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2020-03-19T23:08:30.034 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2020-03-19T23:08:30.034 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2020-03-19T23:08:30.034 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_640.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2020-03-19T23:08:30.034 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2020-03-19T23:08:30.035 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cssselect noarch0.9.2-10.el8 epel 40 k 2020-03-19T23:08:30.035 INFO:teuthology.orchestra.run.smithi038.stdout: python3-html5lib noarch1:0.999999999-6.el8 rhel-8-for-x86_64-appstream-rpms 214 k 2020-03-19T23:08:30.035 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:30.035 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:08:30.035 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:30.035 INFO:teuthology.orchestra.run.smithi038.stdout:Install 32 Packages 2020-03-19T23:08:30.035 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:30.036 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 29 M 2020-03-19T23:08:30.036 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 103 M 2020-03-19T23:08:30.036 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:08:30.172 INFO:teuthology.orchestra.run.smithi038.stdout:(1/32): ceph-15.1.1-234.g787db24.el8.x86_64.rpm 47 kB/s | 6.4 kB 00:00 2020-03-19T23:08:30.416 INFO:teuthology.orchestra.run.smithi038.stdout:(2/32): ceph-mgr-15.1.1-234.g787db24.el8.x86_64 3.7 MB/s | 1.4 MB 00:00 2020-03-19T23:08:30.504 INFO:teuthology.orchestra.run.smithi038.stdout:(3/32): ceph-mds-15.1.1-234.g787db24.el8.x86_64 4.3 MB/s | 2.0 MB 00:00 2020-03-19T23:08:30.558 INFO:teuthology.orchestra.run.smithi038.stdout:(4/32): ceph-mgr-modules-core-15.1.1-234.g787db 3.0 MB/s | 163 kB 00:00 2020-03-19T23:08:30.621 INFO:teuthology.orchestra.run.smithi038.stdout:(5/32): python3-cheroot-8.2.1-1.el8.noarch.rpm 2.3 MB/s | 145 kB 00:00 2020-03-19T23:08:30.642 INFO:teuthology.orchestra.run.smithi038.stdout:(6/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 17 MB/s | 385 kB 00:00 2020-03-19T23:08:30.650 INFO:teuthology.orchestra.run.smithi038.stdout:(7/32): python3-jaraco-6.2-6.el8.noarch.rpm 1.4 MB/s | 11 kB 00:00 2020-03-19T23:08:30.659 INFO:teuthology.orchestra.run.smithi038.stdout:(8/32): python3-jaraco-functools-2.0-4.el8.noar 2.1 MB/s | 18 kB 00:00 2020-03-19T23:08:30.667 INFO:teuthology.orchestra.run.smithi038.stdout:(9/32): python3-logutils-0.3.5-11.el8.noarch.rp 5.7 MB/s | 49 kB 00:00 2020-03-19T23:08:30.695 INFO:teuthology.orchestra.run.smithi038.stdout:(10/32): ceph-mon-15.1.1-234.g787db24.el8.x86_6 8.4 MB/s | 4.4 MB 00:00 2020-03-19T23:08:30.696 INFO:teuthology.orchestra.run.smithi038.stdout:(11/32): python3-more-itertools-7.2.0-3.el8.noa 2.0 MB/s | 59 kB 00:00 2020-03-19T23:08:30.708 INFO:teuthology.orchestra.run.smithi038.stdout:(12/32): python3-portend-2.6-1.el8.noarch.rpm 1.3 MB/s | 16 kB 00:00 2020-03-19T23:08:30.717 INFO:teuthology.orchestra.run.smithi038.stdout:(13/32): python3-simplegeneric-0.8.1-17.el8.noa 2.2 MB/s | 19 kB 00:00 2020-03-19T23:08:30.726 INFO:teuthology.orchestra.run.smithi038.stdout:(14/32): python3-singledispatch-3.4.0.3-18.el8. 2.7 MB/s | 24 kB 00:00 2020-03-19T23:08:30.735 INFO:teuthology.orchestra.run.smithi038.stdout:(15/32): python3-tempora-1.14.1-5.el8.noarch.rp 3.4 MB/s | 30 kB 00:00 2020-03-19T23:08:30.744 INFO:teuthology.orchestra.run.smithi038.stdout:(16/32): python3-trustme-0.5.2-4.el8.noarch.rpm 2.9 MB/s | 27 kB 00:00 2020-03-19T23:08:30.753 INFO:teuthology.orchestra.run.smithi038.stdout:(17/32): python3-zc-lockfile-2.0-2.el8.noarch.r 2.6 MB/s | 23 kB 00:00 2020-03-19T23:08:30.761 INFO:teuthology.orchestra.run.smithi038.stdout:(18/32): python3-pecan-1.3.2-9.el8.noarch.rpm 4.2 MB/s | 283 kB 00:00 2020-03-19T23:08:30.815 INFO:teuthology.orchestra.run.smithi038.stdout:(19/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 705 kB/s | 44 kB 00:00 2020-03-19T23:08:30.845 INFO:teuthology.orchestra.run.smithi038.stdout:(20/32): python3-cssselect-0.9.2-10.el8.noarch. 1.3 MB/s | 40 kB 00:00 2020-03-19T23:08:30.854 INFO:teuthology.orchestra.run.smithi038.stdout:(21/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.9 MB/s | 185 kB 00:00 2020-03-19T23:08:30.892 INFO:teuthology.orchestra.run.smithi038.stdout:(22/32): python3-waitress-1.2.1-2.el8.1.noarch. 5.1 MB/s | 241 kB 00:00 2020-03-19T23:08:30.893 INFO:teuthology.orchestra.run.smithi038.stdout:(23/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 6.3 MB/s | 251 kB 00:00 2020-03-19T23:08:30.909 INFO:teuthology.orchestra.run.smithi038.stdout:(24/32): python3-webtest-2.0.33-1.el8.noarch.rp 5.0 MB/s | 85 kB 00:00 2020-03-19T23:08:30.965 INFO:teuthology.orchestra.run.smithi038.stdout:(25/32): python3-webencodings-0.5.1-6.el8.noarc 376 kB/s | 27 kB 00:00 2020-03-19T23:08:31.010 INFO:teuthology.orchestra.run.smithi038.stdout:(26/32): python3-lxml-4.2.3-1.el8.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2020-03-19T23:08:31.019 INFO:teuthology.orchestra.run.smithi038.stdout:(27/32): python3-werkzeug-0.12.2-4.el8.noarch.r 8.4 MB/s | 457 kB 00:00 2020-03-19T23:08:31.048 INFO:teuthology.orchestra.run.smithi038.stdout:(28/32): python3-mako-1.0.6-13.el8.noarch.rpm 4.0 MB/s | 157 kB 00:00 2020-03-19T23:08:31.073 INFO:teuthology.orchestra.run.smithi038.stdout:(29/32): python3-html5lib-0.999999999-6.el8.noa 4.4 MB/s | 214 kB 00:00 2020-03-19T23:08:31.122 INFO:teuthology.orchestra.run.smithi038.stdout:(30/32): python3-markupsafe-0.23-19.el8.x86_64. 800 kB/s | 39 kB 00:00 2020-03-19T23:08:31.143 INFO:teuthology.orchestra.run.smithi038.stdout:(31/32): python3-pytz-2017.2-9.el8.noarch.rpm 570 kB/s | 54 kB 00:00 2020-03-19T23:08:31.374 INFO:teuthology.orchestra.run.smithi038.stdout:(32/32): ceph-osd-15.1.1-234.g787db24.el8.x86_6 17 MB/s | 17 MB 00:00 2020-03-19T23:08:31.376 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:08:31.377 INFO:teuthology.orchestra.run.smithi038.stdout:Total 22 MB/s | 29 MB 00:01 2020-03-19T23:08:31.398 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:08:31.438 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:08:31.438 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:08:31.795 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:08:31.795 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:08:33.122 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:08:33.315 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 1/32 2020-03-19T23:08:33.507 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 2/32 2020-03-19T23:08:33.779 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 3/32 2020-03-19T23:08:34.012 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 4/32 2020-03-19T23:08:34.245 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 5/32 2020-03-19T23:08:34.481 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/32 2020-03-19T23:08:34.719 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 7/32 2020-03-19T23:08:34.963 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 8/32 2020-03-19T23:08:35.163 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 9/32 2020-03-19T23:08:35.529 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 10/32 2020-03-19T23:08:35.790 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 11/32 2020-03-19T23:08:36.062 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 12/32 2020-03-19T23:08:36.304 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 13/32 2020-03-19T23:08:36.654 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 14/32 2020-03-19T23:08:37.256 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 15/32 2020-03-19T23:08:37.652 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-trustme-0.5.2-4.el8.noarch 16/32 2020-03-19T23:08:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 17/32 2020-03-19T23:08:38.042 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 18/32 2020-03-19T23:08:38.389 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 19/32 2020-03-19T23:08:38.604 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 20/32 2020-03-19T23:08:38.790 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 21/32 2020-03-19T23:08:39.035 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 22/32 2020-03-19T23:08:39.364 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 23/32 2020-03-19T23:08:39.540 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 24/32 2020-03-19T23:08:39.827 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-2.6-1.el8.noarch 25/32 2020-03-19T23:08:40.157 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 26/32 2020-03-19T23:08:40.644 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 27/32 2020-03-19T23:08:40.832 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 28/32 2020-03-19T23:08:42.354 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 28/32 2020-03-19T23:08:42.592 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 29/32 2020-03-19T23:08:43.007 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 29/32 2020-03-19T23:08:43.209 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 30/32 2020-03-19T23:08:43.416 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 30/32 2020-03-19T23:08:43.584 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 31/32 2020-03-19T23:08:43.603 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 31/32 2020-03-19T23:08:43.791 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:15.1.1-234.g787db24.el8.x86_64 32/32 2020-03-19T23:08:44.109 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-2:15.1.1-234.g787db24.el8.x86_64 32/32 2020-03-19T23:08:44.110 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:15.1.1-234.g787db24.el8.x86_64 1/32 2020-03-19T23:08:44.110 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/32 2020-03-19T23:08:44.110 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 3/32 2020-03-19T23:08:44.110 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 4/32 2020-03-19T23:08:44.110 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 5/32 2020-03-19T23:08:44.110 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 6/32 2020-03-19T23:08:44.111 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 7/32 2020-03-19T23:08:44.111 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 8/32 2020-03-19T23:08:44.111 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 9/32 2020-03-19T23:08:44.111 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 10/32 2020-03-19T23:08:44.111 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 11/32 2020-03-19T23:08:44.111 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 12/32 2020-03-19T23:08:44.111 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 13/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-2.6-1.el8.noarch 14/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 15/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 16/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 17/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 18/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 19/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 20/32 2020-03-19T23:08:44.112 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 21/32 2020-03-19T23:08:44.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 22/32 2020-03-19T23:08:44.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 23/32 2020-03-19T23:08:44.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 24/32 2020-03-19T23:08:44.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 25/32 2020-03-19T23:08:44.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 26/32 2020-03-19T23:08:44.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 27/32 2020-03-19T23:08:44.113 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2020-03-19T23:08:44.114 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 29/32 2020-03-19T23:08:44.114 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 30/32 2020-03-19T23:08:44.114 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 31/32 2020-03-19T23:08:46.944 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 32/32 2020-03-19T23:08:46.944 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:08:47.156 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:47.157 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:08:47.157 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:47.157 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-03-19T23:08:47.157 INFO:teuthology.orchestra.run.smithi038.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-03-19T23:08:47.157 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:47.157 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:47.158 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:47.158 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:47.158 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:08:47.158 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-03-19T23:08:47.158 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-03-19T23:08:47.158 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-6.2-6.el8.noarch 2020-03-19T23:08:47.158 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-03-19T23:08:47.159 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-03-19T23:08:47.159 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-03-19T23:08:47.159 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-03-19T23:08:47.159 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-2.6-1.el8.noarch 2020-03-19T23:08:47.159 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-03-19T23:08:47.159 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-03-19T23:08:47.159 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-03-19T23:08:47.160 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-03-19T23:08:47.161 INFO:teuthology.orchestra.run.smithi038.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-03-19T23:08:47.161 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-03-19T23:08:47.161 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.0.6-13.el8.noarch 2020-03-19T23:08:47.161 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2017.2-9.el8.noarch 2020-03-19T23:08:47.161 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-03-19T23:08:47.161 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:47.162 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:08:47.348 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-base 2020-03-19T23:08:47.715 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:08:48.574 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 71 kB/s | 2.8 kB 00:00 2020-03-19T23:08:48.666 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 kB/s | 2.4 kB 00:00 2020-03-19T23:08:49.263 INFO:teuthology.orchestra.run.smithi038.stdout:Package ceph-base-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:08:49.453 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:08:49.455 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:08:49.455 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:08:49.589 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install cephadm 2020-03-19T23:08:49.955 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:08:50.703 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 74 kB/s | 2.8 kB 00:00 2020-03-19T23:08:50.798 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 kB/s | 2.4 kB 00:00 2020-03-19T23:08:51.584 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:08:51.585 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:51.586 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2020-03-19T23:08:51.586 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:51.586 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:08:51.586 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm x86_64 2:15.1.1-234.g787db24.el8 ceph 39 k 2020-03-19T23:08:51.587 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:51.587 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:08:51.587 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:51.587 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2020-03-19T23:08:51.588 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:51.588 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 39 k 2020-03-19T23:08:51.588 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 148 k 2020-03-19T23:08:51.588 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:08:51.749 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm-15.1.1-234.g787db24.el8.x86_64.rpm 241 kB/s | 39 kB 00:00 2020-03-19T23:08:51.750 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:08:51.750 INFO:teuthology.orchestra.run.smithi038.stdout:Total 239 kB/s | 39 kB 00:00 2020-03-19T23:08:51.753 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:08:51.760 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:08:51.761 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:08:51.762 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:08:51.762 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:08:51.816 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:08:54.131 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:08:55.100 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:08:57.458 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:08:57.458 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:08:57.658 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:57.659 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:08:57.659 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:08:57.659 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:57.659 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:08:57.799 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-immutable-object-cache 2020-03-19T23:08:58.176 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:08:58.855 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 73 kB/s | 2.8 kB 00:00 2020-03-19T23:08:58.949 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 kB/s | 2.4 kB 00:00 2020-03-19T23:08:59.749 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:08:59.750 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:59.751 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2020-03-19T23:08:59.751 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:59.751 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:08:59.751 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:15.1.1-234.g787db24.el8 ceph 203 k 2020-03-19T23:08:59.751 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:59.751 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:08:59.751 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:08:59.752 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2020-03-19T23:08:59.752 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:08:59.752 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 203 k 2020-03-19T23:08:59.752 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 743 k 2020-03-19T23:08:59.753 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:09:00.007 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-immutable-object-cache-15.1.1-234.g787db24 790 kB/s | 203 kB 00:00 2020-03-19T23:09:00.008 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:09:00.008 INFO:teuthology.orchestra.run.smithi038.stdout:Total 786 kB/s | 203 kB 00:00 2020-03-19T23:09:00.011 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:09:00.024 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:09:00.024 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:09:00.079 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:09:00.079 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:09:00.216 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:09:00.382 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T23:09:00.730 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T23:09:03.207 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T23:09:03.207 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:09:03.408 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:03.408 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:09:03.409 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:09:03.409 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:03.409 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:03.580 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr 2020-03-19T23:09:03.950 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:04.850 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 76 kB/s | 2.8 kB 00:00 2020-03-19T23:09:04.943 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 kB/s | 2.4 kB 00:00 2020-03-19T23:09:05.652 INFO:teuthology.orchestra.run.smithi038.stdout:Package ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:09:05.842 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:05.843 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:09:05.843 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:06.015 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-dashboard 2020-03-19T23:09:06.384 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:07.055 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 71 kB/s | 2.8 kB 00:00 2020-03-19T23:09:07.148 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 kB/s | 2.4 kB 00:00 2020-03-19T23:09:07.956 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:07.958 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:07.958 INFO:teuthology.orchestra.run.smithi038.stdout: Package ArchVersion Repository Size 2020-03-19T23:09:07.958 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:07.958 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:09:07.958 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch2:15.1.1-234.g787db24.el8 ceph-noarch 3.3 M 2020-03-19T23:09:07.958 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:09:07.958 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch2:15.1.1-234.g787db24.el8 ceph-noarch 22 k 2020-03-19T23:09:07.959 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch0.7-6.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 34 k 2020-03-19T23:09:07.959 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch2.4.1-12.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 197 k 2020-03-19T23:09:07.959 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt noarch1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-03-19T23:09:07.959 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:07.959 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:09:07.959 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:07.959 INFO:teuthology.orchestra.run.smithi038.stdout:Install 5 Packages 2020-03-19T23:09:07.960 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:07.960 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 3.6 M 2020-03-19T23:09:07.960 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 18 M 2020-03-19T23:09:07.960 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:09:08.003 INFO:teuthology.orchestra.run.smithi038.stdout:(1/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 794 kB/s | 34 kB 00:00 2020-03-19T23:09:08.027 INFO:teuthology.orchestra.run.smithi038.stdout:(2/5): python3-routes-2.4.1-12.el8.noarch.rpm 7.9 MB/s | 197 kB 00:00 2020-03-19T23:09:08.068 INFO:teuthology.orchestra.run.smithi038.stdout:(3/5): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2020-03-19T23:09:08.134 INFO:teuthology.orchestra.run.smithi038.stdout:(4/5): ceph-grafana-dashboards-15.1.1-234.g787d 128 kB/s | 22 kB 00:00 2020-03-19T23:09:08.364 INFO:teuthology.orchestra.run.smithi038.stdout:(5/5): ceph-mgr-dashboard-15.1.1-234.g787db24.e 8.3 MB/s | 3.3 MB 00:00 2020-03-19T23:09:08.366 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:09:08.366 INFO:teuthology.orchestra.run.smithi038.stdout:Total 8.9 MB/s | 3.6 MB 00:00 2020-03-19T23:09:08.372 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:09:08.381 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:09:08.382 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:09:08.522 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:09:08.523 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:09:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:09:08.999 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/5 2020-03-19T23:09:09.248 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/5 2020-03-19T23:09:09.438 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/5 2020-03-19T23:09:10.526 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.no 4/5 2020-03-19T23:09:11.505 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 5/5 2020-03-19T23:09:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 5/5 2020-03-19T23:09:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.no 1/5 2020-03-19T23:09:11.738 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 2/5 2020-03-19T23:09:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 3/5 2020-03-19T23:09:11.739 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 4/5 2020-03-19T23:09:14.052 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 5/5 2020-03-19T23:09:14.052 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:09:14.257 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:14.257 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:09:14.257 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:09:14.257 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:09:14.257 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-03-19T23:09:14.257 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.4.1-12.el8.noarch 2020-03-19T23:09:14.258 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-03-19T23:09:14.258 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:14.258 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:14.406 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-diskprediction-local 2020-03-19T23:09:14.778 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:15.774 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 62 kB/s | 2.8 kB 00:00 2020-03-19T23:09:15.878 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 kB/s | 2.4 kB 00:00 2020-03-19T23:09:16.673 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:16.674 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:16.674 INFO:teuthology.orchestra.run.smithi038.stdout: Package ArchVersion Repository Size 2020-03-19T23:09:16.674 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:16.675 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:09:16.675 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch2:15.1.1-234.g787db24.el8 ceph-noarch 8.6 M 2020-03-19T23:09:16.675 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:09:16.675 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads x86_640.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 4.4 M 2020-03-19T23:09:16.675 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_640.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 4.3 M 2020-03-19T23:09:16.675 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_641:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2020-03-19T23:09:16.675 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_641.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_648.3.1-4.5.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_648.3.1-4.5.el8 rhel-8-for-x86_64-baseos-rpms 638 k 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout:Install 8 Packages 2020-03-19T23:09:16.676 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:16.677 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 37 M 2020-03-19T23:09:16.677 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 204 M 2020-03-19T23:09:16.677 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:09:16.817 INFO:teuthology.orchestra.run.smithi038.stdout:(1/8): openblas-threads-0.3.3-2.el8.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2020-03-19T23:09:16.899 INFO:teuthology.orchestra.run.smithi038.stdout:(2/8): openblas-0.3.3-2.el8.x86_64.rpm 19 MB/s | 4.3 MB 00:00 2020-03-19T23:09:16.947 INFO:teuthology.orchestra.run.smithi038.stdout:(3/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 29 MB/s | 3.7 MB 00:00 2020-03-19T23:09:17.034 INFO:teuthology.orchestra.run.smithi038.stdout:(4/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 1.6 MB/s | 225 kB 00:00 2020-03-19T23:09:17.087 INFO:teuthology.orchestra.run.smithi038.stdout:(5/8): libquadmath-8.3.1-4.5.el8.x86_64.rpm 3.1 MB/s | 169 kB 00:00 2020-03-19T23:09:17.138 INFO:teuthology.orchestra.run.smithi038.stdout:(6/8): libgfortran-8.3.1-4.5.el8.x86_64.rpm 12 MB/s | 638 kB 00:00 2020-03-19T23:09:17.259 INFO:teuthology.orchestra.run.smithi038.stdout:(7/8): python3-scipy-1.0.0-20.module+el8.1.0+33 47 MB/s | 15 MB 00:00 2020-03-19T23:09:17.370 INFO:teuthology.orchestra.run.smithi038.stdout:(8/8): ceph-mgr-diskprediction-local-15.1.1-234 12 MB/s | 8.6 MB 00:00 2020-03-19T23:09:17.371 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:09:17.371 INFO:teuthology.orchestra.run.smithi038.stdout:Total 53 MB/s | 37 MB 00:00 2020-03-19T23:09:17.384 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:09:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:09:17.404 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:09:17.726 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:09:17.726 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:09:18.299 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:09:18.505 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-8.3.1-4.5.el8.x86_64 1/8 2020-03-19T23:09:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 1/8 2020-03-19T23:09:18.837 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-8.3.1-4.5.el8.x86_64 2/8 2020-03-19T23:09:19.415 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 2/8 2020-03-19T23:09:19.546 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-threads-0.3.3-2.el8.x86_64 3/8 2020-03-19T23:09:20.153 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 3/8 2020-03-19T23:09:20.337 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.3-2.el8.x86_64 4/8 2020-03-19T23:09:21.104 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 4/8 2020-03-19T23:09:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2020-03-19T23:09:21.662 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-03-19T23:09:23.820 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-03-19T23:09:25.224 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2020-03-19T23:09:25.446 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24. 8/8 2020-03-19T23:09:25.700 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24. 8/8 2020-03-19T23:09:25.701 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24. 1/8 2020-03-19T23:09:25.701 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 2/8 2020-03-19T23:09:25.701 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 3/8 2020-03-19T23:09:25.701 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2020-03-19T23:09:25.701 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2020-03-19T23:09:25.701 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 6/8 2020-03-19T23:09:25.702 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 7/8 2020-03-19T23:09:27.959 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 8/8 2020-03-19T23:09:27.959 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:09:28.171 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:28.172 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:09:28.172 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:09:28.172 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-03-19T23:09:28.172 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.3-2.el8.x86_64 2020-03-19T23:09:28.172 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-03-19T23:09:28.173 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-03-19T23:09:28.173 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2020-03-19T23:09:28.173 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-03-19T23:09:28.173 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-03-19T23:09:28.173 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:28.173 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:28.309 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-rook 2020-03-19T23:09:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:29.435 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 64 kB/s | 2.8 kB 00:00 2020-03-19T23:09:29.532 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 kB/s | 2.4 kB 00:00 2020-03-19T23:09:30.344 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:30.346 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:30.346 INFO:teuthology.orchestra.run.smithi038.stdout: Package ArchVersion Repository Size 2020-03-19T23:09:30.346 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:30.346 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:09:30.347 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch2:15.1.1-234.g787db24.el8 ceph-noarch 25 k 2020-03-19T23:09:30.347 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:09:30.347 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch3.1.1-4.el8 epel 33 k 2020-03-19T23:09:30.347 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch2018.10.15-7.el8 epel 16 k 2020-03-19T23:09:30.347 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch1:1.1.1-10.el8 epel 81 k 2020-03-19T23:09:30.348 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch10.0.1-2.el8 epel 1.1 M 2020-03-19T23:09:30.348 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch3.4.2-12.el8 epel 73 k 2020-03-19T23:09:30.348 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch0.56.0-5.el8 epel 61 k 2020-03-19T23:09:30.348 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-03-19T23:09:30.349 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2020-03-19T23:09:30.349 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2020-03-19T23:09:30.349 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2020-03-19T23:09:30.349 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-03-19T23:09:30.349 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2020-03-19T23:09:30.350 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:30.350 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:09:30.350 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:30.350 INFO:teuthology.orchestra.run.smithi038.stdout:Install 13 Packages 2020-03-19T23:09:30.351 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:30.352 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 1.9 M 2020-03-19T23:09:30.352 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 23 M 2020-03-19T23:09:30.352 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:09:30.396 INFO:teuthology.orchestra.run.smithi038.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 349 kB/s | 16 kB 00:00 2020-03-19T23:09:30.396 INFO:teuthology.orchestra.run.smithi038.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 697 kB/s | 33 kB 00:00 2020-03-19T23:09:30.427 INFO:teuthology.orchestra.run.smithi038.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 2.5 MB/s | 81 kB 00:00 2020-03-19T23:09:30.443 INFO:teuthology.orchestra.run.smithi038.stdout:(4/13): python3-rsa-3.4.2-12.el8.noarch.rpm 4.5 MB/s | 73 kB 00:00 2020-03-19T23:09:30.465 INFO:teuthology.orchestra.run.smithi038.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 3.8 MB/s | 61 kB 00:00 2020-03-19T23:09:30.498 INFO:teuthology.orchestra.run.smithi038.stdout:(6/13): python3-kubernetes-10.0.1-2.el8.noarch. 11 MB/s | 1.1 MB 00:00 2020-03-19T23:09:30.502 INFO:teuthology.orchestra.run.smithi038.stdout:(7/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 623 kB/s | 27 kB 00:00 2020-03-19T23:09:30.521 INFO:teuthology.orchestra.run.smithi038.stdout:(8/13): ceph-mgr-rook-15.1.1-234.g787db24.el8.n 146 kB/s | 25 kB 00:00 2020-03-19T23:09:30.547 INFO:teuthology.orchestra.run.smithi038.stdout:(9/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.7 MB/s | 126 kB 00:00 2020-03-19T23:09:30.569 INFO:teuthology.orchestra.run.smithi038.stdout:(10/13): python3-jsonpointer-1.10-11.el8.noarch 283 kB/s | 20 kB 00:00 2020-03-19T23:09:30.593 INFO:teuthology.orchestra.run.smithi038.stdout:(11/13): python3-requests-oauthlib-1.0.0-1.el8. 935 kB/s | 43 kB 00:00 2020-03-19T23:09:30.602 INFO:teuthology.orchestra.run.smithi038.stdout:(12/13): python3-pyasn1-modules-0.3.7-6.el8.noa 1.3 MB/s | 110 kB 00:00 2020-03-19T23:09:30.610 INFO:teuthology.orchestra.run.smithi038.stdout:(13/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 3.7 MB/s | 155 kB 00:00 2020-03-19T23:09:30.611 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:09:30.611 INFO:teuthology.orchestra.run.smithi038.stdout:Total 7.1 MB/s | 1.9 MB 00:00 2020-03-19T23:09:30.619 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:09:30.635 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:09:30.635 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:09:30.709 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:09:30.710 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:09:31.197 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:09:31.446 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2020-03-19T23:09:31.735 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-3.4.2-12.el8.noarch 2/13 2020-03-19T23:09:32.006 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2020-03-19T23:09:32.210 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2020-03-19T23:09:32.384 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2020-03-19T23:09:32.585 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2020-03-19T23:09:32.766 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2020-03-19T23:09:33.006 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2020-03-19T23:09:33.279 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2020-03-19T23:09:33.559 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2020-03-19T23:09:34.757 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2020-03-19T23:09:35.078 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-10.0.1-2.el8.noarch 12/13 2020-03-19T23:09:35.218 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 13/13 2020-03-19T23:09:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 13/13 2020-03-19T23:09:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/13 2020-03-19T23:09:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2020-03-19T23:09:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2020-03-19T23:09:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2020-03-19T23:09:35.473 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-10.0.1-2.el8.noarch 5/13 2020-03-19T23:09:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-3.4.2-12.el8.noarch 6/13 2020-03-19T23:09:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2020-03-19T23:09:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2020-03-19T23:09:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 9/13 2020-03-19T23:09:35.474 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 10/13 2020-03-19T23:09:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 11/13 2020-03-19T23:09:35.475 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 12/13 2020-03-19T23:09:37.946 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 13/13 2020-03-19T23:09:37.946 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:09:38.153 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:38.153 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:09:38.153 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:09:38.153 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-03-19T23:09:38.154 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-03-19T23:09:38.154 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-03-19T23:09:38.154 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-10.0.1-2.el8.noarch 2020-03-19T23:09:38.154 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-3.4.2-12.el8.noarch 2020-03-19T23:09:38.154 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-03-19T23:09:38.154 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-03-19T23:09:38.155 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-03-19T23:09:38.155 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-03-19T23:09:38.155 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-03-19T23:09:38.155 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-03-19T23:09:38.155 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-03-19T23:09:38.155 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:38.155 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:38.320 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-cephadm 2020-03-19T23:09:38.701 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:39.354 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 67 kB/s | 2.8 kB 00:00 2020-03-19T23:09:39.448 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 kB/s | 2.4 kB 00:00 2020-03-19T23:09:40.255 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:40.256 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:40.256 INFO:teuthology.orchestra.run.smithi038.stdout: Package ArchVersion Repository Size 2020-03-19T23:09:40.256 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:40.257 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:09:40.257 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch2:15.1.1-234.g787db24.el8 ceph-noarch 38 k 2020-03-19T23:09:40.257 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:09:40.257 INFO:teuthology.orchestra.run.smithi038.stdout: python3-execnet noarch1.7.1-1.el8.2 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 356 k 2020-03-19T23:09:40.257 INFO:teuthology.orchestra.run.smithi038.stdout: python3-remoto noarch1.1.4-4.el8 copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 48 k 2020-03-19T23:09:40.257 INFO:teuthology.orchestra.run.smithi038.stdout: python3-apipkg noarch1.5-6.el8 epel 21 k 2020-03-19T23:09:40.257 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:40.258 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:09:40.258 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:40.258 INFO:teuthology.orchestra.run.smithi038.stdout:Install 4 Packages 2020-03-19T23:09:40.258 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:40.259 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 463 k 2020-03-19T23:09:40.259 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 1.4 M 2020-03-19T23:09:40.259 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:09:40.312 INFO:teuthology.orchestra.run.smithi038.stdout:(1/4): python3-remoto-1.1.4-4.el8.noarch.rpm 904 kB/s | 48 kB 00:00 2020-03-19T23:09:40.337 INFO:teuthology.orchestra.run.smithi038.stdout:(2/4): python3-execnet-1.7.1-1.el8.2.noarch.rpm 4.4 MB/s | 356 kB 00:00 2020-03-19T23:09:40.359 INFO:teuthology.orchestra.run.smithi038.stdout:(3/4): python3-apipkg-1.5-6.el8.noarch.rpm 446 kB/s | 21 kB 00:00 2020-03-19T23:09:40.493 INFO:teuthology.orchestra.run.smithi038.stdout:(4/4): ceph-mgr-cephadm-15.1.1-234.g787db24.el8 160 kB/s | 38 kB 00:00 2020-03-19T23:09:40.495 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:09:40.495 INFO:teuthology.orchestra.run.smithi038.stdout:Total 1.9 MB/s | 463 kB 00:00 2020-03-19T23:09:40.501 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:09:40.507 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:09:40.507 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:09:40.541 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:09:40.542 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:09:40.732 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:09:41.060 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/4 2020-03-19T23:09:41.366 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-execnet-1.7.1-1.el8.2.noarch 2/4 2020-03-19T23:09:41.558 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 3/4 2020-03-19T23:09:41.742 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 4/4 2020-03-19T23:09:41.974 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 4/4 2020-03-19T23:09:41.974 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 1/4 2020-03-19T23:09:41.975 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-execnet-1.7.1-1.el8.2.noarch 2/4 2020-03-19T23:09:41.975 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 3/4 2020-03-19T23:09:44.133 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/4 2020-03-19T23:09:44.133 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:09:44.368 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:44.368 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:09:44.368 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:09:44.368 INFO:teuthology.orchestra.run.smithi038.stdout: python3-execnet-1.7.1-1.el8.2.noarch 2020-03-19T23:09:44.369 INFO:teuthology.orchestra.run.smithi038.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-03-19T23:09:44.369 INFO:teuthology.orchestra.run.smithi038.stdout: python3-apipkg-1.5-6.el8.noarch 2020-03-19T23:09:44.369 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:44.369 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:44.512 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-fuse 2020-03-19T23:09:44.891 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:45.757 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 70 kB/s | 2.8 kB 00:00 2020-03-19T23:09:45.870 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 2.4 kB 00:00 2020-03-19T23:09:46.644 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:46.645 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:46.645 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2020-03-19T23:09:46.645 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:46.645 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:09:46.646 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:15.1.1-234.g787db24.el8 ceph 572 k 2020-03-19T23:09:46.646 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2020-03-19T23:09:46.646 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2020-03-19T23:09:46.646 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-common 2020-03-19T23:09:46.646 INFO:teuthology.orchestra.run.smithi038.stdout: x86_64 3.2.1-12.el8 rhel-8-for-x86_64-baseos-rpms 21 k 2020-03-19T23:09:46.646 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:46.646 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:09:46.647 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:46.647 INFO:teuthology.orchestra.run.smithi038.stdout:Install 3 Packages 2020-03-19T23:09:46.647 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:46.647 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 676 k 2020-03-19T23:09:46.648 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 2.1 M 2020-03-19T23:09:46.648 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:09:46.703 INFO:teuthology.orchestra.run.smithi038.stdout:(1/3): fuse-2.9.7-12.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2020-03-19T23:09:46.710 INFO:teuthology.orchestra.run.smithi038.stdout:(2/3): fuse-common-3.2.1-12.el8.x86_64.rpm 335 kB/s | 21 kB 00:00 2020-03-19T23:09:47.027 INFO:teuthology.orchestra.run.smithi038.stdout:(3/3): ceph-fuse-15.1.1-234.g787db24.el8.x86_64 1.5 MB/s | 572 kB 00:00 2020-03-19T23:09:47.028 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:09:47.028 INFO:teuthology.orchestra.run.smithi038.stdout:Total 1.7 MB/s | 676 kB 00:00 2020-03-19T23:09:47.033 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:09:47.055 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:09:47.055 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:09:47.126 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:09:47.127 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:09:47.299 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:09:47.300 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 1/3 2020-03-19T23:09:47.300 INFO:teuthology.orchestra.run.smithi038.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-03-19T23:09:47.300 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:47.727 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T23:09:47.932 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T23:09:48.245 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T23:09:48.245 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T23:09:48.245 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T23:09:51.056 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T23:09:51.056 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:09:51.302 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:51.302 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:09:51.302 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2020-03-19T23:09:51.302 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-03-19T23:09:51.302 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:51.303 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:51.452 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install librados-devel 2020-03-19T23:09:51.834 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:52.596 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 69 kB/s | 2.8 kB 00:00 2020-03-19T23:09:52.692 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 kB/s | 2.4 kB 00:00 2020-03-19T23:09:53.483 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:53.484 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2020-03-19T23:09:53.484 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:53.484 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:09:53.484 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:15.1.1-234.g787db24.el8 ceph 141 k 2020-03-19T23:09:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:53.485 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:09:53.485 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:09:53.485 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2020-03-19T23:09:53.485 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:53.486 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 141 k 2020-03-19T23:09:53.486 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 487 k 2020-03-19T23:09:53.486 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:09:53.771 INFO:teuthology.orchestra.run.smithi038.stdout:librados-devel-15.1.1-234.g787db24.el8.x86_64.r 492 kB/s | 141 kB 00:00 2020-03-19T23:09:53.772 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:09:53.772 INFO:teuthology.orchestra.run.smithi038.stdout:Total 490 kB/s | 141 kB 00:00 2020-03-19T23:09:53.775 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:09:53.782 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:09:53.782 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:09:53.837 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:09:53.838 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:09:53.953 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:09:54.122 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:09:54.345 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:09:56.695 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:09:56.695 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:09:56.917 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:56.917 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:09:56.917 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:09:56.917 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:09:56.918 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:57.232 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install libcephfs2 2020-03-19T23:09:57.611 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:09:58.241 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 68 kB/s | 2.8 kB 00:00 2020-03-19T23:09:58.335 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 kB/s | 2.4 kB 00:00 2020-03-19T23:09:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:Package libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:09:59.119 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:09:59.121 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:09:59.121 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:09:59.255 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install libcephfs-devel 2020-03-19T23:09:59.633 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:00.554 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 kB/s | 2.8 kB 00:00 2020-03-19T23:10:00.658 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 kB/s | 2.4 kB 00:00 2020-03-19T23:10:01.443 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:01.444 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:01.444 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2020-03-19T23:10:01.444 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:01.444 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:10:01.445 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:15.1.1-234.g787db24.el8 ceph 21 k 2020-03-19T23:10:01.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:01.445 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:10:01.445 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:01.445 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2020-03-19T23:10:01.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:01.446 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 21 k 2020-03-19T23:10:01.446 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 76 k 2020-03-19T23:10:01.446 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:10:01.683 INFO:teuthology.orchestra.run.smithi038.stdout:libcephfs-devel-15.1.1-234.g787db24.el8.x86_64. 89 kB/s | 21 kB 00:00 2020-03-19T23:10:01.684 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:10:01.684 INFO:teuthology.orchestra.run.smithi038.stdout:Total 89 kB/s | 21 kB 00:00 2020-03-19T23:10:01.687 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:10:01.691 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:10:01.691 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:10:01.693 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:10:01.693 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:10:01.746 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:10:01.955 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:02.163 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:04.217 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:04.217 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:10:04.425 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:04.425 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:10:04.426 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:10:04.426 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:04.426 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:04.605 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install librados2 2020-03-19T23:10:04.985 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:05.797 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 73 kB/s | 2.8 kB 00:00 2020-03-19T23:10:05.895 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 55 kB/s | 2.4 kB 00:00 2020-03-19T23:10:06.494 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:10:06.685 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:06.686 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:10:06.686 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:06.816 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install librbd1 2020-03-19T23:10:07.195 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:07.875 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 73 kB/s | 2.8 kB 00:00 2020-03-19T23:10:07.969 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 kB/s | 2.4 kB 00:00 2020-03-19T23:10:08.561 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:10:08.751 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:08.752 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:10:08.752 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:08.886 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rados 2020-03-19T23:10:09.268 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:09.896 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 72 kB/s | 2.8 kB 00:00 2020-03-19T23:10:09.989 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 kB/s | 2.4 kB 00:00 2020-03-19T23:10:10.585 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rados-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:10:10.775 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:10.776 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:10:10.777 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:10.911 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rgw 2020-03-19T23:10:11.290 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:12.025 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 72 kB/s | 2.8 kB 00:00 2020-03-19T23:10:12.122 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 kB/s | 2.4 kB 00:00 2020-03-19T23:10:12.718 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:10:12.909 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:12.910 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:10:12.910 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:13.048 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-cephfs 2020-03-19T23:10:13.426 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:14.186 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 kB/s | 2.8 kB 00:00 2020-03-19T23:10:14.282 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 kB/s | 2.4 kB 00:00 2020-03-19T23:10:14.926 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:10:15.115 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:15.159 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:10:15.159 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:15.299 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rbd 2020-03-19T23:10:15.678 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:16.281 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 72 kB/s | 2.8 kB 00:00 2020-03-19T23:10:16.371 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 kB/s | 2.4 kB 00:00 2020-03-19T23:10:16.967 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 is already installed. 2020-03-19T23:10:17.156 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:17.157 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:10:17.158 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:17.487 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-fuse 2020-03-19T23:10:17.864 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:18.565 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 62 kB/s | 2.8 kB 00:00 2020-03-19T23:10:18.658 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 kB/s | 2.4 kB 00:00 2020-03-19T23:10:19.444 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:19.445 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:19.445 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2020-03-19T23:10:19.445 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:19.446 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:10:19.446 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:15.1.1-234.g787db24.el8 ceph 72 k 2020-03-19T23:10:19.446 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:19.446 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:10:19.447 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:19.447 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2020-03-19T23:10:19.447 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:19.447 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 72 k 2020-03-19T23:10:19.448 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 230 k 2020-03-19T23:10:19.448 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:10:19.645 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-fuse-15.1.1-234.g787db24.el8.x86_64.rpm 362 kB/s | 72 kB 00:00 2020-03-19T23:10:19.646 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:10:19.647 INFO:teuthology.orchestra.run.smithi038.stdout:Total 359 kB/s | 72 kB 00:00 2020-03-19T23:10:19.649 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:10:19.661 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:10:19.661 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:10:19.717 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:10:19.717 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:10:19.843 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:10:20.060 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:20.284 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:22.391 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:10:22.596 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:22.596 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:10:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:10:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:22.597 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:22.749 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-mirror 2020-03-19T23:10:23.128 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:23.775 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 74 kB/s | 2.8 kB 00:00 2020-03-19T23:10:23.888 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 2.4 kB 00:00 2020-03-19T23:10:24.682 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:24.683 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:24.683 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2020-03-19T23:10:24.683 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:24.683 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:10:24.683 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:15.1.1-234.g787db24.el8 ceph 2.7 M 2020-03-19T23:10:24.683 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:24.684 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:10:24.684 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:24.684 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2020-03-19T23:10:24.684 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:24.685 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 2.7 M 2020-03-19T23:10:24.685 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 13 M 2020-03-19T23:10:24.685 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:10:25.177 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-mirror-15.1.1-234.g787db24.el8.x86_64.rpm 5.5 MB/s | 2.7 MB 00:00 2020-03-19T23:10:25.178 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:10:25.178 INFO:teuthology.orchestra.run.smithi038.stdout:Total 5.4 MB/s | 2.7 MB 00:00 2020-03-19T23:10:25.181 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:10:25.195 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:10:25.195 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:10:25.260 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:10:25.260 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:10:25.639 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:10:25.870 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:26.187 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:28.366 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:28.366 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:10:28.571 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:28.571 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:10:28.572 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:10:28.572 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:28.572 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:28.724 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-nbd 2020-03-19T23:10:29.106 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:10:29.905 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 70 kB/s | 2.8 kB 00:00 2020-03-19T23:10:30.023 INFO:teuthology.orchestra.run.smithi038.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 kB/s | 2.4 kB 00:00 2020-03-19T23:10:30.821 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:10:30.821 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:30.822 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2020-03-19T23:10:30.822 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:30.822 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2020-03-19T23:10:30.822 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:15.1.1-234.g787db24.el8 ceph 141 k 2020-03-19T23:10:30.822 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:30.823 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:10:30.823 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:10:30.823 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2020-03-19T23:10:30.823 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:30.824 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 141 k 2020-03-19T23:10:30.824 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 526 k 2020-03-19T23:10:30.825 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2020-03-19T23:10:31.145 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-nbd-15.1.1-234.g787db24.el8.x86_64.rpm 438 kB/s | 141 kB 00:00 2020-03-19T23:10:31.145 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2020-03-19T23:10:31.146 INFO:teuthology.orchestra.run.smithi038.stdout:Total 436 kB/s | 141 kB 00:00 2020-03-19T23:10:31.148 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:10:31.160 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:10:31.160 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:10:31.216 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:10:31.216 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:10:31.341 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:10:31.534 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:31.743 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:33.860 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:10:33.861 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:10:34.071 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:34.071 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2020-03-19T23:10:34.071 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:10:34.071 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:34.071 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:10:34.249 DEBUG:teuthology.parallel:result is None 2020-03-19T23:10:34.249 INFO:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2020-03-19T23:10:34.290 INFO:teuthology.orchestra.run.smithi038.stdout:RedHatEnterprise 2020-03-19T23:10:34.292 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T23:10:34.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=787db24c312ed9d310cd3f194b00548ede9a199c 2020-03-19T23:10:34.516 INFO:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-03-19T23:10:34.540 INFO:teuthology.orchestra.run.smithi038.stdout:15.1.1-234.g787db24.el8 2020-03-19T23:10:34.541 INFO:teuthology.packaging:The installed version of ceph is 15.1.1-234.g787db24.el8 2020-03-19T23:10:34.541 INFO:teuthology.task.install:The correct ceph version 15.1.1-234.g787db24 is installed. 2020-03-19T23:10:34.541 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-03-19T23:10:34.542 INFO:teuthology.orchestra.run.smithi038:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-03-19T23:10:34.615 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-03-19T23:10:34.615 INFO:teuthology.orchestra.run.smithi038:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-03-19T23:10:34.688 INFO:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-03-19T23:10:34.760 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-03-19T23:10:34.760 INFO:teuthology.orchestra.run.smithi038:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-03-19T23:10:34.845 INFO:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-03-19T23:10:34.915 INFO:teuthology.run_tasks:Running task ceph... 2020-03-19T23:10:34.987 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-03-19T23:10:34.988 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 777 /var/log/ceph 2020-03-19T23:10:35.019 INFO:tasks.ceph:Disabling ceph logrotate... 2020-03-19T23:10:35.019 INFO:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/logrotate.d/ceph 2020-03-19T23:10:35.090 INFO:tasks.ceph:Creating extra log directories... 2020-03-19T23:10:35.090 INFO:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-03-19T23:10:35.161 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-03-19T23:10:35.161 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'debug ms': 1, 'debug mds': 20, 'osd max object namespace len': 64, 'osd max object name len': 460, 'debug client': 20, 'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}}} 2020-03-19T23:10:35.161 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (1, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'ceph', 'pre-mgr-commands': ['sudo ceph config set mgr mgr/devicehealth/enable_monitoring false --force'], 'log-whitelist': ['overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'conf': {'global': {'debug ms': 1, 'debug mds': 20, 'osd max object namespace len': 64, 'osd max object name len': 460, 'debug client': 20, 'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}}, 'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}}, {'exec': {'client.0': ['ceph osd pool create data_cache 4', 'ceph osd tier add cephfs_data data_cache', 'ceph osd tier cache-mode data_cache writeback', 'ceph osd tier set-overlay cephfs_data data_cache', 'ceph osd pool set data_cache hit_set_type bloom', 'ceph osd pool set data_cache hit_set_count 8', 'ceph osd pool set data_cache hit_set_period 3600', 'ceph osd pool set data_cache min_read_recency_for_promote 0']}}, {'ceph-fuse': None}, {'exec': {'client.0': ['sudo chmod 777 $TESTDIR/mnt.0/', 'dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5', 'ls -al $TESTDIR/mnt.0/foo', 'truncate --size 0 $TESTDIR/mnt.0/foo', 'ls -al $TESTDIR/mnt.0/foo', 'dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5', 'ls -al $TESTDIR/mnt.0/foo', 'cp $TESTDIR/mnt.0/foo /tmp/foo', 'sync', 'rados -p data_cache ls -', 'sleep 10', 'rados -p data_cache ls -', 'rados -p data_cache cache-flush-evict-all', 'rados -p data_cache ls -', 'sleep 1']}}, {'exec': {'client.1': ['hexdump -C /tmp/foo | head', 'hexdump -C $TESTDIR/mnt.1/foo | head', 'cmp $TESTDIR/mnt.1/foo /tmp/foo']}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.16490', 'owner': 'scheduled_sage@teuthology', 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'targets': {'smithi038.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXXrJ1MD0XA8HHO9piq5w48uyQLNujW4uBAQHZiAGFfP7tgFrzj5spaWB1hOIQNBperOdNaHJJ7Wtkx4tx4LJsHQsUwmqJzGXtuhulWyI3Bopr8elLzlhK22CTzhmOKF4GKIHiNrn/Aa/SwdQpuwslqGSeXkll44a+qI2VISu+JuQUlGnC5ENbWRPvpYUjGvapFlEcSoDVeZQQIBrWV+45ej0waj7cpzEyFsaNicXrixbNucm2hkbSOUYcqHba7JqkVqQZrfLRKcoRXV1c+87XoJaS47ZvToi8vUGsJRi10sdH4T+RU1oAJCXNoOKHXSDl46BwlU2zxXzEsn6pm8oR/7mziFAfF+PYIm6eKNdw1ka0c+KD3Y1E0PU5FcpCE8jjhXZWCQR34pGRO4m832oydY0cNSNy1cu1jNGdB8Vii5Qq1+v+YZyoar/t5/q30PIVxMXzHu3FkTqPvGUWh5IDo4BitrxdQRigPiVZk6hI8usXxbuZbfrrRnj6H4a3FNU='}, 'job_id': '4868731', 'tube': 'smithi', 'suite_branch': 'wip-sage-testing-2020-03-19-0855', 'priority': 90, 'os_version': '8.1', 'first_in_suite': False, 'branch': 'wip-sage-testing-2020-03-19-0855', 'suite': 'rados', 'email': None, 'archive_path': '/home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731', 'description': 'rados/singleton-nomsgr/{all/cache-fs-trunc.yaml mon_election/classic.yaml rados.yaml supported-random-distro$/{rhel_8.yaml}}', 'nuke-on-error': True, 'repo': 'git://git.ceph.com/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'name': 'sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi', 'roles': [['mon.a', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'os_type': 'rhel', 'overrides': {'ceph': {'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'conf': {'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'debug ms': 1}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}}, 'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c'}}, 'admin_socket': {'branch': 'wip-sage-testing-2020-03-19-0855'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'sha1': '787db24c312ed9d310cd3f194b00548ede9a199c', 'branch': 'wip-sage-testing-2020-03-19-0855'}}} 2020-03-19T23:10:35.162 INFO:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-03-19T23:10:35.215 INFO:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/run/ceph 2020-03-19T23:10:35.344 DEBUG:teuthology.orchestra.remote:smithi038:/scratch_devs is 72B 2020-03-19T23:10:35.392 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2020-03-19T23:10:35.393 INFO:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-03-19T23:10:35.407 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-1 2020-03-19T23:10:35.408 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T23:10:35.408 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 59239 Links: 1 2020-03-19T23:10:35.408 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T23:10:35.408 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T23:10:35.408 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2020-03-19 23:10:26.009839692 +0000 2020-03-19T23:10:35.409 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2020-03-19 23:07:08.023051326 +0000 2020-03-19T23:10:35.409 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2020-03-19 23:07:08.023051326 +0000 2020-03-19T23:10:35.409 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2020-03-19T23:10:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2020-03-19T23:10:35.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2020-03-19T23:10:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000223507 s, 2.3 MB/s 2020-03-19T23:10:35.429 INFO:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-03-19T23:10:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-03-19T23:10:35.484 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T23:10:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 62128 Links: 1 2020-03-19T23:10:35.484 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T23:10:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T23:10:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2020-03-19 23:10:26.009839692 +0000 2020-03-19T23:10:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2020-03-19 23:07:08.378044358 +0000 2020-03-19T23:10:35.485 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2020-03-19 23:07:08.378044358 +0000 2020-03-19T23:10:35.485 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2020-03-19T23:10:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2020-03-19T23:10:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2020-03-19T23:10:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000291278 s, 1.8 MB/s 2020-03-19T23:10:35.503 INFO:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-03-19T23:10:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-3 2020-03-19T23:10:35.558 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T23:10:35.558 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 65006 Links: 1 2020-03-19T23:10:35.558 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T23:10:35.559 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T23:10:35.559 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2020-03-19 23:10:26.009839692 +0000 2020-03-19T23:10:35.559 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2020-03-19 23:07:08.748037094 +0000 2020-03-19T23:10:35.559 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2020-03-19 23:07:08.748037094 +0000 2020-03-19T23:10:35.559 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2020-03-19T23:10:35.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2020-03-19T23:10:35.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2020-03-19T23:10:35.575 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000266878 s, 1.9 MB/s 2020-03-19T23:10:35.578 INFO:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-03-19T23:10:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-4 2020-03-19T23:10:35.632 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-03-19T23:10:35.632 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 60065 Links: 1 2020-03-19T23:10:35.633 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-03-19T23:10:35.633 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2020-03-19T23:10:35.633 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2020-03-19 23:10:26.009839692 +0000 2020-03-19T23:10:35.633 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2020-03-19 23:07:09.123029732 +0000 2020-03-19T23:10:35.633 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2020-03-19 23:07:09.123029732 +0000 2020-03-19T23:10:35.633 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2020-03-19T23:10:35.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2020-03-19T23:10:35.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2020-03-19T23:10:35.649 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000287092 s, 1.8 MB/s 2020-03-19T23:10:35.651 INFO:tasks.ceph:fs option selected, checking for scratch devs 2020-03-19T23:10:35.652 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2020-03-19T23:10:35.652 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2020-03-19T23:10:35.652 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T23:10:35.652 INFO:tasks.ceph:Generating config... 2020-03-19T23:10:35.653 INFO:tasks.ceph:[global] debug ms = 1 2020-03-19T23:10:35.653 INFO:tasks.ceph:[global] debug mds = 20 2020-03-19T23:10:35.653 INFO:tasks.ceph:[global] osd max object namespace len = 64 2020-03-19T23:10:35.654 INFO:tasks.ceph:[global] osd max object name len = 460 2020-03-19T23:10:35.654 INFO:tasks.ceph:[global] debug client = 20 2020-03-19T23:10:35.654 INFO:tasks.ceph:[global] mon election default strategy = 1 2020-03-19T23:10:35.654 INFO:tasks.ceph:[mon] debug mon = 20 2020-03-19T23:10:35.654 INFO:tasks.ceph:[mon] debug paxos = 20 2020-03-19T23:10:35.654 INFO:tasks.ceph:[mon] debug ms = 1 2020-03-19T23:10:35.654 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2020-03-19T23:10:35.654 INFO:tasks.ceph:[osd] debug ms = 20 2020-03-19T23:10:35.655 INFO:tasks.ceph:[osd] debug journal = 20 2020-03-19T23:10:35.655 INFO:tasks.ceph:[osd] debug osd = 25 2020-03-19T23:10:35.655 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2020-03-19T23:10:35.655 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2020-03-19T23:10:35.655 INFO:tasks.ceph:[osd] osd op queue = debug_random 2020-03-19T23:10:35.655 INFO:tasks.ceph:[osd] debug filestore = 20 2020-03-19T23:10:35.655 INFO:tasks.ceph:Setting up mon.a... 2020-03-19T23:10:35.656 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-03-19T23:10:35.741 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.keyring 2020-03-19T23:10:35.743 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-03-19T23:10:35.788 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-03-19T23:10:35.819 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.38')] 2020-03-19T23:10:35.819 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon host': '172.21.15.38', 'debug ms': 1, 'debug mds': 20, 'osd max object namespace len': 64, 'osd max object name len': 460, 'debug client': 20, 'mon election default strategy': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd op queue cut off': 'debug_random', 'debug ms': 20, 'debug journal': 20, 'debug osd': 25, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd op queue': 'debug_random', 'debug filestore': 20}, 'mgr': {'debug ms': '1', 'debug mgr': '20', 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mds.a': {}} 2020-03-19T23:10:35.821 INFO:teuthology.orchestra.run.smithi038:> cat > /home/ubuntu/cephtest/ceph.tmp.conf 2020-03-19T23:10:35.873 INFO:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.38 --print /home/ubuntu/cephtest/ceph.monmap 2020-03-19T23:10:35.948 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-03-19T23:10:35.948 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: generated fsid a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3 2020-03-19T23:10:35.948 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 0 2020-03-19T23:10:35.948 INFO:teuthology.orchestra.run.smithi038.stdout:fsid a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3 2020-03-19T23:10:35.948 INFO:teuthology.orchestra.run.smithi038.stdout:last_changed 2020-03-19T23:10:35.947465+0000 2020-03-19T23:10:35.948 INFO:teuthology.orchestra.run.smithi038.stdout:created 2020-03-19T23:10:35.947465+0000 2020-03-19T23:10:35.949 INFO:teuthology.orchestra.run.smithi038.stdout:min_mon_release 0 (unknown) 2020-03-19T23:10:35.949 INFO:teuthology.orchestra.run.smithi038.stdout:election_strategy: 1 2020-03-19T23:10:35.949 INFO:teuthology.orchestra.run.smithi038.stdout:0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2020-03-19T23:10:35.949 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2020-03-19T23:10:35.950 INFO:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-03-19T23:10:36.005 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3... 2020-03-19T23:10:36.006 INFO:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-03-19T23:10:36.109 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2020-03-19T23:10:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: chdir = "" 2020-03-19T23:10:36.109 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-03-19T23:10:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: auth supported = cephx 2020-03-19T23:10:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: filestore xattr use omap = true 2020-03-19T23:10:36.110 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: mon clock drift allowed = 1.000 2020-03-19T23:10:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: osd crush chooseleaf type = 0 2020-03-19T23:10:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: auth debug = true 2020-03-19T23:10:36.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on old message = true 2020-03-19T23:10:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on bug = true 2020-03-19T23:10:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.112 INFO:teuthology.orchestra.run.smithi038.stdout: mon max pg per osd = 10000 # >= luminous 2020-03-19T23:10:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: mon pg warn max object skew = 0 2020-03-19T23:10:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: # disable pg_autoscaler by default for new pools 2020-03-19T23:10:36.113 INFO:teuthology.orchestra.run.smithi038.stdout: osd_pool_default_pg_autoscale_mode = off 2020-03-19T23:10:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default size = 2 2020-03-19T23:10:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.114 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow primary affinity = true 2020-03-19T23:10:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow pg remap = true 2020-03-19T23:10:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on legacy crush tunables = false 2020-03-19T23:10:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on crush straw calc version zero = false 2020-03-19T23:10:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on no sortbitwise = false 2020-03-19T23:10:36.115 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on osd down out interval zero = false 2020-03-19T23:10:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on too few osds = false 2020-03-19T23:10:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-03-19T23:10:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_no_redundancy = false 2020-03-19T23:10:36.116 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-03-19T23:10:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: osd default data pool replay window = 5 2020-03-19T23:10:36.117 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: mon allow pool delete = true 2020-03-19T23:10:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: mon cluster log file level = debug 2020-03-19T23:10:36.118 INFO:teuthology.orchestra.run.smithi038.stdout: debug asserts on shutdown = true 2020-03-19T23:10:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: mon host = 172.21.15.38 2020-03-19T23:10:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2020-03-19T23:10:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: debug mds = 20 2020-03-19T23:10:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: osd max object namespace len = 64 2020-03-19T23:10:36.119 INFO:teuthology.orchestra.run.smithi038.stdout: osd max object name len = 460 2020-03-19T23:10:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: debug client = 20 2020-03-19T23:10:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: mon election default strategy = 1 2020-03-19T23:10:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3 2020-03-19T23:10:36.120 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.121 INFO:teuthology.orchestra.run.smithi038.stdout:[osd] 2020-03-19T23:10:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: osd journal size = 100 2020-03-19T23:10:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.121 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub load threshold = 5.0 2020-03-19T23:10:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub max interval = 600 2020-03-19T23:10:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: osd recover clone overlap = true 2020-03-19T23:10:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: osd recovery max chunk = 1048576 2020-03-19T23:10:36.122 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug shutdown = true 2020-03-19T23:10:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug op order = true 2020-03-19T23:10:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug verify stray on activate = true 2020-03-19T23:10:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: osd open classes on start = true 2020-03-19T23:10:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug pg log writeout = true 2020-03-19T23:10:36.123 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: osd deep scrub update digest min age = 30 2020-03-19T23:10:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: osd map max advance = 10 2020-03-19T23:10:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: journal zero on create = true 2020-03-19T23:10:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.124 INFO:teuthology.orchestra.run.smithi038.stdout: filestore ondisk finisher threads = 3 2020-03-19T23:10:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: filestore apply finisher threads = 3 2020-03-19T23:10:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: bdev debug aio = true 2020-03-19T23:10:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug misdirected ops = true 2020-03-19T23:10:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: osd op queue cut off = debug_random 2020-03-19T23:10:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 20 2020-03-19T23:10:36.125 INFO:teuthology.orchestra.run.smithi038.stdout: debug journal = 20 2020-03-19T23:10:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: debug osd = 25 2020-03-19T23:10:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug verify cached snaps = True 2020-03-19T23:10:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug verify missing on start = True 2020-03-19T23:10:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: osd op queue = debug_random 2020-03-19T23:10:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: debug filestore = 20 2020-03-19T23:10:36.126 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.126 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr] 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgr = 20 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: mgr/telemetry/nag = false 2020-03-19T23:10:36.127 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.128 INFO:teuthology.orchestra.run.smithi038.stdout:[mon] 2020-03-19T23:10:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2020-03-19T23:10:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2020-03-19T23:10:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: debug paxos = 20 2020-03-19T23:10:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2020-03-19T23:10:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: mon data avail warn = 5 2020-03-19T23:10:36.128 INFO:teuthology.orchestra.run.smithi038.stdout: mon mgr mkfs grace = 240 2020-03-19T23:10:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2020-03-19T23:10:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd reporter subtree level = osd 2020-03-19T23:10:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd prime pg temp = true 2020-03-19T23:10:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2020-03-19T23:10:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:10:36.129 INFO:teuthology.orchestra.run.smithi038.stdout:[client] 2020-03-19T23:10:36.129 INFO:teuthology.orchestra.run.smithi038.stdout: rgw cache enabled = true 2020-03-19T23:10:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable ops log = true 2020-03-19T23:10:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable usage log = true 2020-03-19T23:10:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-03-19T23:10:36.130 INFO:teuthology.orchestra.run.smithi038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-03-19T23:10:36.130 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.a] 2020-03-19T23:10:36.130 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2020-03-19T23:10:36.131 INFO:tasks.ceph:Creating admin key on mon.a... 2020-03-19T23:10:36.131 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-03-19T23:10:36.187 INFO:tasks.ceph:Copying monmap to all nodes... 2020-03-19T23:10:36.207 DEBUG:teuthology.orchestra.remote:smithi038:/etc/ceph/ceph.keyring is 206B 2020-03-19T23:10:36.303 DEBUG:teuthology.orchestra.remote:smithi038:/home/ubuntu/cephtest/ceph.monmap is 188B 2020-03-19T23:10:36.352 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:10:36.353 INFO:teuthology.orchestra.run.smithi038:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-03-19T23:10:36.416 INFO:teuthology.orchestra.run.smithi038:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-03-19T23:10:36.500 INFO:tasks.ceph:Setting up mon nodes... 2020-03-19T23:10:36.501 INFO:tasks.ceph:Setting up mgr nodes... 2020-03-19T23:10:36.501 INFO:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-03-19T23:10:36.601 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-03-19T23:10:36.603 INFO:tasks.ceph:Setting up mds nodes... 2020-03-19T23:10:36.604 INFO:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-03-19T23:10:36.664 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-03-19T23:10:36.667 INFO:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-03-19T23:10:36.698 INFO:tasks.ceph_client:Setting up client nodes... 2020-03-19T23:10:36.699 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-03-19T23:10:36.782 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-03-19T23:10:36.801 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-03-19T23:10:36.884 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-03-19T23:10:36.903 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-03-19T23:10:36.904 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'}} 2020-03-19T23:10:36.904 INFO:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-03-19T23:10:36.975 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T23:10:36.975 INFO:tasks.ceph:{} 2020-03-19T23:10:36.976 INFO:tasks.ceph:osd.0 2020-03-19T23:10:36.976 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:10:36.976 INFO:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-03-19T23:10:38.362 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-03-19T23:10:38.362 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2020-03-19T23:10:38.363 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-03-19T23:10:38.363 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2020-03-19T23:10:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-03-19T23:10:38.363 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2020-03-19T23:10:38.363 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-03-19T23:10:38.364 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-03-19T23:10:38.364 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-03-19T23:10:38.364 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-03-19T23:10:38.365 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2020-03-19T23:10:38.366 INFO:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2020-03-19T23:10:38.418 INFO:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-03-19T23:10:38.454 INFO:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-03-19T23:10:38.525 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T23:10:38.526 INFO:tasks.ceph:{} 2020-03-19T23:10:38.526 INFO:tasks.ceph:osd.1 2020-03-19T23:10:38.526 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:10:38.526 INFO:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-03-19T23:10:39.908 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-03-19T23:10:39.909 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2020-03-19T23:10:39.909 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-03-19T23:10:39.909 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2020-03-19T23:10:39.909 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-03-19T23:10:39.909 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2020-03-19T23:10:39.909 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-03-19T23:10:39.910 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-03-19T23:10:39.910 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-03-19T23:10:39.910 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-03-19T23:10:39.911 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2020-03-19T23:10:39.911 INFO:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2020-03-19T23:10:39.960 INFO:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-03-19T23:10:39.996 INFO:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-03-19T23:10:40.068 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.2': '/dev/vg_nvme/lv_2'} 2020-03-19T23:10:40.069 INFO:tasks.ceph:{} 2020-03-19T23:10:40.069 INFO:tasks.ceph:osd.2 2020-03-19T23:10:40.069 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:10:40.069 INFO:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-03-19T23:10:41.462 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-03-19T23:10:41.462 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2020-03-19T23:10:41.462 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-03-19T23:10:41.462 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2020-03-19T23:10:41.462 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-03-19T23:10:41.463 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2020-03-19T23:10:41.463 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-03-19T23:10:41.463 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-03-19T23:10:41.463 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-03-19T23:10:41.463 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-03-19T23:10:41.466 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2020-03-19T23:10:41.466 INFO:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2020-03-19T23:10:41.515 INFO:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-03-19T23:10:41.551 INFO:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-03-19T23:10:41.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:41.637+0000 7fbdc3131ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-03-19T23:10:41.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:41.637+0000 7fbdc3131ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-03-19T23:10:41.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:41.638+0000 7fbdc3131ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-03-19T23:10:41.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:41.638+0000 7fbdc3131ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-03-19T23:10:41.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:41.638+0000 7fbdc3131ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-03-19T23:10:41.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:41.638+0000 7fbdc3131ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-03-19T23:10:43.481 INFO:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-03-19T23:10:43.517 INFO:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-03-19T23:10:43.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:43.607+0000 7f57eb8fbec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-03-19T23:10:43.610 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:43.608+0000 7f57eb8fbec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-03-19T23:10:43.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:43.608+0000 7f57eb8fbec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-03-19T23:10:43.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:43.608+0000 7f57eb8fbec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-03-19T23:10:43.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:43.608+0000 7f57eb8fbec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-03-19T23:10:43.612 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:43.609+0000 7f57eb8fbec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-03-19T23:10:45.448 INFO:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-03-19T23:10:45.485 INFO:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-03-19T23:10:45.578 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:45.576+0000 7fa2c79a3ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-03-19T23:10:45.578 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:45.576+0000 7fa2c79a3ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-03-19T23:10:45.579 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:45.576+0000 7fa2c79a3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-03-19T23:10:45.579 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:45.576+0000 7fa2c79a3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-03-19T23:10:45.579 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:45.576+0000 7fa2c79a3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-03-19T23:10:45.579 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:45.577+0000 7fa2c79a3ec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-03-19T23:10:47.413 INFO:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-03-19T23:10:47.449 INFO:tasks.ceph:Reading keys from all nodes... 2020-03-19T23:10:47.450 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:47.506 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.3IueCiS5na 2020-03-19T23:10:47.507 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.3IueCiS5na 2020-03-19T23:10:47.583 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.3IueCiS5na 2020-03-19T23:10:47.715 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.3IueCiS5na is 56B 2020-03-19T23:10:47.763 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.3IueCiS5na 2020-03-19T23:10:47.778 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:47.833 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.MDT5bAwoaS 2020-03-19T23:10:47.834 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.MDT5bAwoaS 2020-03-19T23:10:47.908 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.MDT5bAwoaS 2020-03-19T23:10:48.039 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.MDT5bAwoaS is 56B 2020-03-19T23:10:48.088 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.MDT5bAwoaS 2020-03-19T23:10:48.103 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:48.157 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.Luw4PDLgK9 2020-03-19T23:10:48.157 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.Luw4PDLgK9 2020-03-19T23:10:48.230 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.Luw4PDLgK9 2020-03-19T23:10:48.361 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.Luw4PDLgK9 is 56B 2020-03-19T23:10:48.409 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.Luw4PDLgK9 2020-03-19T23:10:48.425 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:48.478 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.p4oR7wDNII 2020-03-19T23:10:48.479 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.p4oR7wDNII 2020-03-19T23:10:48.551 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.p4oR7wDNII 2020-03-19T23:10:48.680 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.p4oR7wDNII is 56B 2020-03-19T23:10:48.730 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.p4oR7wDNII 2020-03-19T23:10:48.747 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:48.800 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.oXnIz2MtLH 2020-03-19T23:10:48.801 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.oXnIz2MtLH 2020-03-19T23:10:48.874 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.oXnIz2MtLH 2020-03-19T23:10:49.005 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.oXnIz2MtLH is 56B 2020-03-19T23:10:49.068 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.oXnIz2MtLH 2020-03-19T23:10:49.139 DEBUG:teuthology.orchestra.remote:smithi038:/etc/ceph/ceph.client.0.keyring is 59B 2020-03-19T23:10:49.248 DEBUG:teuthology.orchestra.remote:smithi038:/etc/ceph/ceph.client.1.keyring is 59B 2020-03-19T23:10:49.299 INFO:tasks.ceph:Adding keys to all mons... 2020-03-19T23:10:49.300 INFO:teuthology.orchestra.run.smithi038:> sudo tee -a /etc/ceph/ceph.keyring 2020-03-19T23:10:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.x] 2020-03-19T23:10:49.351 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDs+3NeT8/SIxAAx3To6U08zl8umHFbWtFvVQ== 2020-03-19T23:10:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2020-03-19T23:10:49.352 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDs+3NeB8KVJxAA93wfXtXsihJXgyhNqcHjWQ== 2020-03-19T23:10:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.0] 2020-03-19T23:10:49.352 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDx+3NeSv8WJhAA7yS+5TMWB6Ok2Abb3MmP0Q== 2020-03-19T23:10:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.1] 2020-03-19T23:10:49.352 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDz+3Nern5PJBAAmvuE8k7PNWtY68Kg6G2YfA== 2020-03-19T23:10:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.2] 2020-03-19T23:10:49.353 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQD1+3NeOgxuIhAAR2NZTtdGjp795crI9PMlLg== 2020-03-19T23:10:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2020-03-19T23:10:49.353 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDs+3NeAGGYLhAAQSMD386jWtaG44/KaJRXeg== 2020-03-19T23:10:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:[client.1] 2020-03-19T23:10:49.353 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDs+3Ne9OCwNBAAcGVnrXSsy7prmeHZFSBfKA== 2020-03-19T23:10:49.354 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2020-03-19T23:10:49.462 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mds allow --cap osd 'allow *' --cap mgr 'allow *' --cap mon 'allow *' 2020-03-19T23:10:49.525 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-03-19T23:10:49.579 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-03-19T23:10:49.634 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2020-03-19T23:10:49.687 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-03-19T23:10:49.748 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2020-03-19T23:10:49.806 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-03-19T23:10:49.806 INFO:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-03-19T23:10:49.839 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-03-19T23:10:49.943 INFO:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-03-19T23:10:49.982 INFO:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-03-19T23:10:50.042 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-03-19T23:10:50.043 INFO:tasks.ceph.mon.a:Restarting daemon 2020-03-19T23:10:50.043 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-03-19T23:10:50.085 INFO:tasks.ceph.mon.a:Started 2020-03-19T23:10:50.086 INFO:teuthology.orchestra.run.smithi038:> sudo ceph config set mgr mgr/devicehealth/enable_monitoring false --force 2020-03-19T23:10:50.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.207+0000 7f38b856c700 1 Processor -- start 2020-03-19T23:10:50.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.207+0000 7f38b856c700 1 -- start start 2020-03-19T23:10:50.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.207+0000 7f38b856c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f38b007e780 0x7f38b008f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:50.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.207+0000 7f38b856c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b005c700 con 0x7f38b008f680 2020-03-19T23:10:50.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.207+0000 7f38b856c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b0090520 con 0x7f38b007e780 2020-03-19T23:10:50.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f38b007e780 0x7f38b008f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:50.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6308700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f38b008f680 0x7f38b008fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43596/0 (socket says 172.21.15.38:43596) 2020-03-19T23:10:50.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6308700 1 -- 172.21.15.38:0/1238713663 learned_addr learned my addr 172.21.15.38:0/1238713663 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:50.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 -- 172.21.15.38:0/1238713663 >> v1:172.21.15.38:6789/0 conn(0x7f38b008f680 legacy=0x7f38b008fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 -- 172.21.15.38:0/1238713663 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b007eb80 con 0x7f38b007e780 2020-03-19T23:10:50.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b5b07700 1 -- 172.21.15.38:0/1238713663 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743872723 0 0) 0x7f38b005c700 con 0x7f38b008f680 2020-03-19T23:10:50.214 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 --2- 172.21.15.38:0/1238713663 >> v2:172.21.15.38:3300/0 conn(0x7f38b007e780 0x7f38b008f220 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f38ac002f30 tx=0x7f38ac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.214 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 --2- 172.21.15.38:0/1238713663 >> v2:172.21.15.38:3300/0 conn(0x7f38b007e780 0x7f38b008f220 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f38ac002f30 tx=0x7f38ac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.214 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 --2- 172.21.15.38:0/1238713663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b007e780 0x7f38b008f220 secure :-1 s=READY pgs=2 cs=0 l=1 rx=0x7f38ac002f30 tx=0x7f38ac007a40).ready entity=mon.0 client_cookie=d394b673c25d830d server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:50.214 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 --2- 172.21.15.38:0/1238713663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b007e780 0x7f38b008f220 secure :-1 s=THROTTLE_DONE pgs=2 cs=0 l=1 rx=0x7f38ac002f30 tx=0x7f38ac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.214 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 --2- 172.21.15.38:0/1238713663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b007e780 0x7f38b008f220 secure :-1 s=THROTTLE_DONE pgs=2 cs=0 l=1 rx=0x7f38ac002f30 tx=0x7f38ac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.214 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b5b07700 1 -- 172.21.15.38:0/1238713663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f38ac0113f0 con 0x7f38b007e780 2020-03-19T23:10:50.215 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b6b09700 1 --2- 172.21.15.38:0/1238713663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b007e780 0x7f38b008f220 secure :-1 s=THROTTLE_DONE pgs=2 cs=0 l=1 rx=0x7f38ac002f30 tx=0x7f38ac007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.215 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b5b07700 1 -- 172.21.15.38:0/1238713663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac011550 con 0x7f38b007e780 2020-03-19T23:10:50.215 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b5b07700 1 -- 172.21.15.38:0/1238713663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f38ac011830 con 0x7f38b007e780 2020-03-19T23:10:50.215 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b856c700 1 -- 172.21.15.38:0/1238713663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b007e780 msgr2=0x7f38b008f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.215 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.208+0000 7f38b856c700 1 --2- 172.21.15.38:0/1238713663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b007e780 0x7f38b008f220 secure :-1 s=READY pgs=2 cs=0 l=1 rx=0x7f38ac002f30 tx=0x7f38ac007a40).stop 2020-03-19T23:10:50.215 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.209+0000 7f38b856c700 1 -- 172.21.15.38:0/1238713663 shutdown_connections 2020-03-19T23:10:50.216 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.209+0000 7f38b856c700 1 --2- 172.21.15.38:0/1238713663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b007e780 0x7f38b008f220 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:50.216 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.209+0000 7f38b856c700 1 -- 172.21.15.38:0/1238713663 >> 172.21.15.38:0/1238713663 conn(0x7f38b0095460 msgr2=0x7f38b0091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:50.216 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.209+0000 7f38b856c700 1 -- 172.21.15.38:0/1238713663 shutdown_connections 2020-03-19T23:10:50.216 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.209+0000 7f38b856c700 1 -- 172.21.15.38:0/1238713663 wait complete. 2020-03-19T23:10:50.216 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b856c700 1 Processor -- start 2020-03-19T23:10:50.216 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b856c700 1 -- start start 2020-03-19T23:10:50.224 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b856c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f38b008f680 0x7f38b0074210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:50.225 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b856c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b0092f60 con 0x7f38b007e780 2020-03-19T23:10:50.225 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b856c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b007dd00 con 0x7f38b008f680 2020-03-19T23:10:50.225 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b6b09700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f38b007e780 0x7f38b0072ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43598/0 (socket says 172.21.15.38:43598) 2020-03-19T23:10:50.225 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b6b09700 1 -- 172.21.15.38:0/1943199714 learned_addr learned my addr 172.21.15.38:0/1943199714 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:50.225 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> v2:172.21.15.38:3300/0 conn(0x7f38b008f680 0x7f38b0074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:50.225 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645342129 0 0) 0x7f38b0092f60 con 0x7f38b007e780 2020-03-19T23:10:50.226 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.210+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3898003230 con 0x7f38b007e780 2020-03-19T23:10:50.226 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 -- 172.21.15.38:0/1943199714 >> v1:172.21.15.38:6789/0 conn(0x7f38b007e780 legacy=0x7f38b0072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.226 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 -- 172.21.15.38:0/1943199714 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00755e0 con 0x7f38b008f680 2020-03-19T23:10:50.226 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> v2:172.21.15.38:3300/0 conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.226 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> v2:172.21.15.38:3300/0 conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.227 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=READY pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:50.227 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.227 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f38a0005b20 con 0x7f38b008f680 2020-03-19T23:10:50.227 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.227 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.227 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0005c80 con 0x7f38b008f680 2020-03-19T23:10:50.228 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b0093b10 con 0x7f38b008f680 2020-03-19T23:10:50.228 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f38a00060c0 con 0x7f38b008f680 2020-03-19T23:10:50.228 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b0093200 con 0x7f38b008f680 2020-03-19T23:10:50.228 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.228 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f38a000a6e0 con 0x7f38b008f680 2020-03-19T23:10:50.228 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.229 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.211+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f38a0009930 con 0x7f38b008f680 2020-03-19T23:10:50.229 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.212+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b0056db0 con 0x7f38b008f680 2020-03-19T23:10:50.229 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.214+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.229 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.214+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f38a00090b0 con 0x7f38b008f680 2020-03-19T23:10:50.292 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.290+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config set", "who": "mgr", "name": "mgr/devicehealth/enable_monitoring", "value": "false", "force": true} v 0) v1 -- 0x7f38b00607f0 con 0x7f38b008f680 2020-03-19T23:10:50.296 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.294+0000 7f38b6308700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=THROTTLE_DONE pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.296 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.294+0000 7f38a77fe700 1 -- 172.21.15.38:0/1943199714 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mgr", "name": "mgr/devicehealth/enable_monitoring", "value": "false", "force": true}]=0 v2) v1 ==== 151+0+0 (secure 0 0 0) 0x7f38a0009bd0 con 0x7f38b008f680 2020-03-19T23:10:50.297 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.294+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 msgr2=0x7f38b0074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.297 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.294+0000 7f38b856c700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 secure :-1 s=READY pgs=4 cs=0 l=1 rx=0x7f38a0000b80 tx=0x7f38a00108a0).stop 2020-03-19T23:10:50.297 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.295+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 shutdown_connections 2020-03-19T23:10:50.297 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.295+0000 7f38b856c700 1 --2- 172.21.15.38:0/1943199714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38b008f680 0x7f38b0074210 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:50.297 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.295+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 >> 172.21.15.38:0/1943199714 conn(0x7f38b0095460 msgr2=0x7f38b00635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.295+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 shutdown_connections 2020-03-19T23:10:50.298 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.295+0000 7f38b856c700 1 -- 172.21.15.38:0/1943199714 wait complete. 2020-03-19T23:10:50.306 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-03-19T23:10:50.307 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-03-19T23:10:50.307 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-03-19T23:10:50.311 INFO:tasks.ceph.mgr.x:Started 2020-03-19T23:10:50.311 INFO:tasks.ceph:Setting crush tunables to default 2020-03-19T23:10:50.311 INFO:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd crush tunables default 2020-03-19T23:10:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.422+0000 7f889b7af700 1 Processor -- start 2020-03-19T23:10:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.422+0000 7f889b7af700 1 -- start start 2020-03-19T23:10:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889b7af700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894087cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889b7af700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8894090f70 con 0x7f889408bce0 2020-03-19T23:10:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889b7af700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88940904e0 con 0x7f889408c7d0 2020-03-19T23:10:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894087cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f8899d4c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f889408bce0 0x7f889408c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43610/0 (socket says 172.21.15.38:43610) 2020-03-19T23:10:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894087cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60470/0 (socket says 172.21.15.38:60470) 2020-03-19T23:10:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f8899d4c700 1 -- 172.21.15.38:0/409338807 learned_addr learned my addr 172.21.15.38:0/409338807 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 -- 172.21.15.38:0/409338807 >> v1:172.21.15.38:6789/0 conn(0x7f889408bce0 legacy=0x7f889408c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 -- 172.21.15.38:0/409338807 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940908e0 con 0x7f889408c7d0 2020-03-19T23:10:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f8898d4a700 1 -- 172.21.15.38:0/409338807 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342380890 0 0) 0x7f8894090f70 con 0x7f889408bce0 2020-03-19T23:10:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- 172.21.15.38:0/409338807 >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894087cb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f8890002f30 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- 172.21.15.38:0/409338807 >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894087cb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f8890002f30 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- 172.21.15.38:0/409338807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894087cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rx=0x7f8890002f30 tx=0x7f8890007a40).ready entity=mon.0 client_cookie=4fefc78bce6dab10 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- 172.21.15.38:0/409338807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894087cb0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7f8890002f30 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f8898d4a700 1 -- 172.21.15.38:0/409338807 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8890008e80 con 0x7f889408c7d0 2020-03-19T23:10:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- 172.21.15.38:0/409338807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894087cb0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7f8890002f30 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889954b700 1 --2- 172.21.15.38:0/409338807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894087cb0 secure :-1 s=THROTTLE_DONE pgs=10 cs=0 l=1 rx=0x7f8890002f30 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f8898d4a700 1 -- 172.21.15.38:0/409338807 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88900113f0 con 0x7f889408c7d0 2020-03-19T23:10:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f8898d4a700 1 -- 172.21.15.38:0/409338807 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f88900116d0 con 0x7f889408c7d0 2020-03-19T23:10:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889b7af700 1 -- 172.21.15.38:0/409338807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 msgr2=0x7f8894087cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.423+0000 7f889b7af700 1 --2- 172.21.15.38:0/409338807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894087cb0 secure :-1 s=READY pgs=10 cs=0 l=1 rx=0x7f8890002f30 tx=0x7f8890007a40).stop 2020-03-19T23:10:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.424+0000 7f889b7af700 1 -- 172.21.15.38:0/409338807 shutdown_connections 2020-03-19T23:10:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.424+0000 7f889b7af700 1 --2- 172.21.15.38:0/409338807 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894087cb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:50.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.424+0000 7f889b7af700 1 -- 172.21.15.38:0/409338807 >> 172.21.15.38:0/409338807 conn(0x7f88940951d0 msgr2=0x7f8894091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.424+0000 7f889b7af700 1 -- 172.21.15.38:0/409338807 shutdown_connections 2020-03-19T23:10:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.424+0000 7f889b7af700 1 -- 172.21.15.38:0/409338807 wait complete. 2020-03-19T23:10:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.425+0000 7f889b7af700 1 Processor -- start 2020-03-19T23:10:50.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.425+0000 7f889b7af700 1 -- start start 2020-03-19T23:10:50.434 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889b7af700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889b7af700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8894074b60 con 0x7f889408bce0 2020-03-19T23:10:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889b7af700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8894073720 con 0x7f889408c7d0 2020-03-19T23:10:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f8899d4c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f889408bce0 0x7f8894071c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43614/0 (socket says 172.21.15.38:43614) 2020-03-19T23:10:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f8899d4c700 1 -- 172.21.15.38:0/1518586476 learned_addr learned my addr 172.21.15.38:0/1518586476 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779425054 0 0) 0x7f8894074b60 con 0x7f889408bce0 2020-03-19T23:10:50.435 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f887c003230 con 0x7f889408bce0 2020-03-19T23:10:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 -- 172.21.15.38:0/1518586476 >> v1:172.21.15.38:6789/0 conn(0x7f889408bce0 legacy=0x7f8894071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 -- 172.21.15.38:0/1518586476 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894074e70 con 0x7f889408c7d0 2020-03-19T23:10:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> v2:172.21.15.38:3300/0 conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=READY pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.436 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f8890011e30 con 0x7f889408c7d0 2020-03-19T23:10:50.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8890007150 con 0x7f889408c7d0 2020-03-19T23:10:50.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f88900073f0 con 0x7f889408c7d0 2020-03-19T23:10:50.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.426+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8894074850 con 0x7f889408c7d0 2020-03-19T23:10:50.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.427+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.427+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f8890005a80 con 0x7f889408c7d0 2020-03-19T23:10:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.427+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894073d30 con 0x7f889408c7d0 2020-03-19T23:10:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.427+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.427+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f889000cb40 con 0x7f889408c7d0 2020-03-19T23:10:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.428+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894056db0 con 0x7f889408c7d0 2020-03-19T23:10:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.430+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.430+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f889000cd00 con 0x7f889408c7d0 2020-03-19T23:10:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.511+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f889405e820 con 0x7f889408c7d0 2020-03-19T23:10:50.517 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.515+0000 7f889954b700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=THROTTLE_DONE pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.515+0000 7f88867fc700 1 -- 172.21.15.38:0/1518586476 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f889000d400 con 0x7f889408c7d0 2020-03-19T23:10:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.516+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 msgr2=0x7f8894072380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.516+0000 7f889b7af700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 secure :-1 s=READY pgs=12 cs=0 l=1 rx=0x7f8894072cf0 tx=0x7f8890007a40).stop 2020-03-19T23:10:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.517+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 shutdown_connections 2020-03-19T23:10:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.517+0000 7f889b7af700 1 --2- 172.21.15.38:0/1518586476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f889408c7d0 0x7f8894072380 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:50.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.517+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 >> 172.21.15.38:0/1518586476 conn(0x7f88940951d0 msgr2=0x7f8894062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:50.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.517+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 shutdown_connections 2020-03-19T23:10:50.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:50.517+0000 7f889b7af700 1 -- 172.21.15.38:0/1518586476 wait complete. 2020-03-19T23:10:50.520 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2020-03-19T23:10:50.529 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-03-19T23:10:50.529 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:50.543 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.FjK3wqOeDG 2020-03-19T23:10:50.544 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.FjK3wqOeDG 2020-03-19T23:10:50.615 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.FjK3wqOeDG 2020-03-19T23:10:50.745 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.FjK3wqOeDG is 37B 2020-03-19T23:10:50.792 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.FjK3wqOeDG 2020-03-19T23:10:50.807 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:50.860 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.Hz1q8hI313 2020-03-19T23:10:50.860 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.Hz1q8hI313 2020-03-19T23:10:50.931 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.Hz1q8hI313 2020-03-19T23:10:51.059 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.Hz1q8hI313 is 37B 2020-03-19T23:10:51.107 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.Hz1q8hI313 2020-03-19T23:10:51.129 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:10:51.176 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.2fuwyjabeh 2020-03-19T23:10:51.176 INFO:teuthology.orchestra.run.smithi038:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.2fuwyjabeh 2020-03-19T23:10:51.248 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.2fuwyjabeh 2020-03-19T23:10:51.377 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.2fuwyjabeh is 37B 2020-03-19T23:10:51.413 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.2fuwyjabeh 2020-03-19T23:10:51.450 INFO:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd new e499998d-82cd-4828-9cef-fc02ff95bbd2 0 2020-03-19T23:10:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a7a81d700 1 Processor -- start 2020-03-19T23:10:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a7a81d700 1 -- start start 2020-03-19T23:10:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a7a81d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a7408bce0 0x7f6a7408c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a7a81d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a74090f70 con 0x7f6a7408c520 2020-03-19T23:10:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a7a81d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a740904e0 con 0x7f6a7408bce0 2020-03-19T23:10:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a78dba700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a7408bce0 0x7f6a7408c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a6bfff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6a7408c520 0x7f6a74086ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43618/0 (socket says 172.21.15.38:43618) 2020-03-19T23:10:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.602+0000 7f6a6bfff700 1 -- 172.21.15.38:0/1052143847 learned_addr learned my addr 172.21.15.38:0/1052143847 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 -- 172.21.15.38:0/1052143847 >> v1:172.21.15.38:6789/0 conn(0x7f6a7408c520 legacy=0x7f6a74086ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 -- 172.21.15.38:0/1052143847 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740908e0 con 0x7f6a7408bce0 2020-03-19T23:10:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 --2- 172.21.15.38:0/1052143847 >> v2:172.21.15.38:3300/0 conn(0x7f6a7408bce0 0x7f6a7408c0c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6a70002f30 tx=0x7f6a70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 --2- 172.21.15.38:0/1052143847 >> v2:172.21.15.38:3300/0 conn(0x7f6a7408bce0 0x7f6a7408c0c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6a70002f30 tx=0x7f6a70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 --2- 172.21.15.38:0/1052143847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408bce0 0x7f6a7408c0c0 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f6a70002f30 tx=0x7f6a70007a40).ready entity=mon.0 client_cookie=a05df13a6d9b1a73 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:51.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 --2- 172.21.15.38:0/1052143847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408bce0 0x7f6a7408c0c0 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7f6a70002f30 tx=0x7f6a70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a6b7fe700 1 -- 172.21.15.38:0/1052143847 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6a70008e80 con 0x7f6a7408bce0 2020-03-19T23:10:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 --2- 172.21.15.38:0/1052143847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408bce0 0x7f6a7408c0c0 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7f6a70002f30 tx=0x7f6a70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a78dba700 1 --2- 172.21.15.38:0/1052143847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408bce0 0x7f6a7408c0c0 secure :-1 s=THROTTLE_DONE pgs=14 cs=0 l=1 rx=0x7f6a70002f30 tx=0x7f6a70007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a6b7fe700 1 -- 172.21.15.38:0/1052143847 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a700113f0 con 0x7f6a7408bce0 2020-03-19T23:10:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a6b7fe700 1 -- 172.21.15.38:0/1052143847 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6a70011710 con 0x7f6a7408bce0 2020-03-19T23:10:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a7a81d700 1 -- 172.21.15.38:0/1052143847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408bce0 msgr2=0x7f6a7408c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.609 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.603+0000 7f6a7a81d700 1 --2- 172.21.15.38:0/1052143847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408bce0 0x7f6a7408c0c0 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f6a70002f30 tx=0x7f6a70007a40).stop 2020-03-19T23:10:51.610 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.604+0000 7f6a7a81d700 1 -- 172.21.15.38:0/1052143847 shutdown_connections 2020-03-19T23:10:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.604+0000 7f6a7a81d700 1 --2- 172.21.15.38:0/1052143847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408bce0 0x7f6a7408c0c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.604+0000 7f6a7a81d700 1 -- 172.21.15.38:0/1052143847 >> 172.21.15.38:0/1052143847 conn(0x7f6a740951d0 msgr2=0x7f6a74091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.604+0000 7f6a7a81d700 1 -- 172.21.15.38:0/1052143847 shutdown_connections 2020-03-19T23:10:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.604+0000 7f6a7a81d700 1 -- 172.21.15.38:0/1052143847 wait complete. 2020-03-19T23:10:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a7a81d700 1 Processor -- start 2020-03-19T23:10:51.611 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a7a81d700 1 -- start start 2020-03-19T23:10:51.614 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a7a81d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a7408c520 0x7f6a74074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:51.614 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a7a81d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a741198a0 con 0x7f6a7408bce0 2020-03-19T23:10:51.615 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a7a81d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a74073b10 con 0x7f6a7408c520 2020-03-19T23:10:51.615 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a78dba700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6a7408bce0 0x7f6a7411fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43622/0 (socket says 172.21.15.38:43622) 2020-03-19T23:10:51.615 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a78dba700 1 -- 172.21.15.38:0/838503841 learned_addr learned my addr 172.21.15.38:0/838503841 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:51.616 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> v2:172.21.15.38:3300/0 conn(0x7f6a7408c520 0x7f6a74074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:51.616 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.605+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874204947 0 0) 0x7f6a741198a0 con 0x7f6a7408bce0 2020-03-19T23:10:51.616 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a54003230 con 0x7f6a7408bce0 2020-03-19T23:10:51.616 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 -- 172.21.15.38:0/838503841 >> v1:172.21.15.38:6789/0 conn(0x7f6a7408bce0 legacy=0x7f6a7411fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.616 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 -- 172.21.15.38:0/838503841 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a74119b40 con 0x7f6a7408c520 2020-03-19T23:10:51.616 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> v2:172.21.15.38:3300/0 conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.617 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> v2:172.21.15.38:3300/0 conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.617 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:51.617 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6a64005b20 con 0x7f6a7408c520 2020-03-19T23:10:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a64005c80 con 0x7f6a7408c520 2020-03-19T23:10:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a74072ce0 con 0x7f6a7408c520 2020-03-19T23:10:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6a640060c0 con 0x7f6a7408c520 2020-03-19T23:10:51.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a74120530 con 0x7f6a7408c520 2020-03-19T23:10:51.619 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.619 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.606+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f6a6400a6e0 con 0x7f6a7408c520 2020-03-19T23:10:51.619 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.607+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.619 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.607+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f6a6400a970 con 0x7f6a7408c520 2020-03-19T23:10:51.619 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.607+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a74056db0 con 0x7f6a7408c520 2020-03-19T23:10:51.619 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.609+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.620 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.609+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f6a640090b0 con 0x7f6a7408c520 2020-03-19T23:10:51.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.701+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e499998d-82cd-4828-9cef-fc02ff95bbd2", "id": 0} v 0) v1 -- 0x7f6a74068e90 con 0x7f6a7408c520 2020-03-19T23:10:51.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.707+0000 7f6a6bfff700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=THROTTLE_DONE pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.707+0000 7f6a697fa700 1 -- 172.21.15.38:0/838503841 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e499998d-82cd-4828-9cef-fc02ff95bbd2", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6a64006950 con 0x7f6a7408c520 2020-03-19T23:10:51.709 INFO:teuthology.orchestra.run.smithi038.stdout:0 2020-03-19T23:10:51.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.708+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 msgr2=0x7f6a74074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.708+0000 7f6a7a81d700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f6a74094640 tx=0x7f6a64010810).stop 2020-03-19T23:10:51.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.708+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 shutdown_connections 2020-03-19T23:10:51.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.708+0000 7f6a7a81d700 1 --2- 172.21.15.38:0/838503841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a7408c520 0x7f6a74074270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:51.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.708+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 >> 172.21.15.38:0/838503841 conn(0x7f6a740951d0 msgr2=0x7f6a740935e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:51.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.708+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 shutdown_connections 2020-03-19T23:10:51.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.708+0000 7f6a7a81d700 1 -- 172.21.15.38:0/838503841 wait complete. 2020-03-19T23:10:51.720 INFO:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd new 28063113-c054-454f-8f2a-d4525cfb896a 1 2020-03-19T23:10:51.835 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.830+0000 7fab05ad1700 1 Processor -- start 2020-03-19T23:10:51.835 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.830+0000 7fab05ad1700 1 -- start start 2020-03-19T23:10:51.835 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7fab05ad1700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab0008c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7fab05ad1700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab00090f70 con 0x7fab0008c520 2020-03-19T23:10:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7fab05ad1700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab000904e0 con 0x7fab0008bce0 2020-03-19T23:10:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faafffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab0008c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faaf7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fab0008c520 0x7fab00086ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43626/0 (socket says 172.21.15.38:43626) 2020-03-19T23:10:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faaf7fff700 1 -- 172.21.15.38:0/547413482 learned_addr learned my addr 172.21.15.38:0/547413482 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faafffff700 1 -- 172.21.15.38:0/547413482 >> v1:172.21.15.38:6789/0 conn(0x7fab0008c520 legacy=0x7fab00086ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faafffff700 1 -- 172.21.15.38:0/547413482 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000908e0 con 0x7fab0008bce0 2020-03-19T23:10:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faafffff700 1 --2- 172.21.15.38:0/547413482 >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab0008c0c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7faaf0002f30 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faafffff700 1 --2- 172.21.15.38:0/547413482 >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab0008c0c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7faaf0002f30 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.831+0000 7faafffff700 1 --2- 172.21.15.38:0/547413482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab0008c0c0 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7faaf0002f30 tx=0x7faaf0007a40).ready entity=mon.0 client_cookie=d09fed7424c06604 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7faafffff700 1 --2- 172.21.15.38:0/547413482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab0008c0c0 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7faaf0002f30 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7faaff7fe700 1 -- 172.21.15.38:0/547413482 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7faaf0008e80 con 0x7fab0008bce0 2020-03-19T23:10:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7faafffff700 1 --2- 172.21.15.38:0/547413482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab0008c0c0 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7faaf0002f30 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7faaff7fe700 1 -- 172.21.15.38:0/547413482 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf00113f0 con 0x7fab0008bce0 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7faafffff700 1 --2- 172.21.15.38:0/547413482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab0008c0c0 secure :-1 s=THROTTLE_DONE pgs=18 cs=0 l=1 rx=0x7faaf0002f30 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7faaff7fe700 1 -- 172.21.15.38:0/547413482 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7faaf0011710 con 0x7fab0008bce0 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7fab05ad1700 1 -- 172.21.15.38:0/547413482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 msgr2=0x7fab0008c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7fab05ad1700 1 --2- 172.21.15.38:0/547413482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab0008c0c0 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7faaf0002f30 tx=0x7faaf0007a40).stop 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7fab05ad1700 1 -- 172.21.15.38:0/547413482 shutdown_connections 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7fab05ad1700 1 --2- 172.21.15.38:0/547413482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab0008c0c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7fab05ad1700 1 -- 172.21.15.38:0/547413482 >> 172.21.15.38:0/547413482 conn(0x7fab000951d0 msgr2=0x7fab00091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7fab05ad1700 1 -- 172.21.15.38:0/547413482 shutdown_connections 2020-03-19T23:10:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.832+0000 7fab05ad1700 1 -- 172.21.15.38:0/547413482 wait complete. 2020-03-19T23:10:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.833+0000 7fab05ad1700 1 Processor -- start 2020-03-19T23:10:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.833+0000 7fab05ad1700 1 -- start start 2020-03-19T23:10:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7fab05ad1700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab00085280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7fab05ad1700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0008a1f0 con 0x7fab0008c520 2020-03-19T23:10:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7fab05ad1700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab00093190 con 0x7fab0008bce0 2020-03-19T23:10:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab00085280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faaf7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fab0008c520 0x7fab00088000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43632/0 (socket says 172.21.15.38:43632) 2020-03-19T23:10:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faaf7fff700 1 -- 172.21.15.38:0/1995665969 learned_addr learned my addr 172.21.15.38:0/1995665969 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 -- 172.21.15.38:0/1995665969 >> v1:172.21.15.38:6789/0 conn(0x7fab0008c520 legacy=0x7fab00088000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 -- 172.21.15.38:0/1995665969 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab00089820 con 0x7fab0008bce0 2020-03-19T23:10:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157279 0 0) 0x7fab0008a1f0 con 0x7fab0008c520 2020-03-19T23:10:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> v2:172.21.15.38:3300/0 conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7faaf000f830 con 0x7fab0008bce0 2020-03-19T23:10:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf0007150 con 0x7fab0008bce0 2020-03-19T23:10:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0008e980 con 0x7fab0008bce0 2020-03-19T23:10:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.834+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7faaf00073f0 con 0x7fab0008bce0 2020-03-19T23:10:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.835+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab00093910 con 0x7fab0008bce0 2020-03-19T23:10:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.835+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.835+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.835+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7faaf00058a0 con 0x7fab0008bce0 2020-03-19T23:10:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.835+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7faaf000dbc0 con 0x7fab0008bce0 2020-03-19T23:10:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.835+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab00056db0 con 0x7fab0008bce0 2020-03-19T23:10:51.846 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.837+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.846 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.837+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7faaf000de20 con 0x7fab0008bce0 2020-03-19T23:10:51.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.929+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd new", "uuid": "28063113-c054-454f-8f2a-d4525cfb896a", "id": 1} v 0) v1 -- 0x7fab00089a20 con 0x7fab0008bce0 2020-03-19T23:10:51.935 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.933+0000 7faafffff700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=THROTTLE_DONE pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:51.935 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.933+0000 7faafd7fa700 1 -- 172.21.15.38:0/1995665969 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "28063113-c054-454f-8f2a-d4525cfb896a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7faaf000c330 con 0x7fab0008bce0 2020-03-19T23:10:51.935 INFO:teuthology.orchestra.run.smithi038.stdout:1 2020-03-19T23:10:51.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.934+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 msgr2=0x7fab00085280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:51.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.934+0000 7fab05ad1700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7fab0008a9a0 tx=0x7faaf0007a40).stop 2020-03-19T23:10:51.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.934+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 shutdown_connections 2020-03-19T23:10:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.934+0000 7fab05ad1700 1 --2- 172.21.15.38:0/1995665969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab0008bce0 0x7fab00085280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.934+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 >> 172.21.15.38:0/1995665969 conn(0x7fab000951d0 msgr2=0x7fab00060860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.934+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 shutdown_connections 2020-03-19T23:10:51.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:51.934+0000 7fab05ad1700 1 -- 172.21.15.38:0/1995665969 wait complete. 2020-03-19T23:10:51.946 INFO:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd new f4403c1d-0ebc-4a48-9e62-aa667c0c7f36 2 2020-03-19T23:10:52.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c72a3f700 1 Processor -- start 2020-03-19T23:10:52.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c72a3f700 1 -- start start 2020-03-19T23:10:52.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c72a3f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:52.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c72a3f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c090f70 con 0x7f4c6c08bce0 2020-03-19T23:10:52.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c72a3f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c0904e0 con 0x7f4c6c08c7d0 2020-03-19T23:10:52.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c6bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:52.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c70fdc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4c6c08bce0 0x7f4c6c08c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43634/0 (socket says 172.21.15.38:43634) 2020-03-19T23:10:52.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c70fdc700 1 -- 172.21.15.38:0/305581774 learned_addr learned my addr 172.21.15.38:0/305581774 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:52.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c6bfff700 1 -- 172.21.15.38:0/305581774 >> v1:172.21.15.38:6789/0 conn(0x7f4c6c08bce0 legacy=0x7f4c6c08c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6bfff700 1 -- 172.21.15.38:0/305581774 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0908e0 con 0x7f4c6c08c7d0 2020-03-19T23:10:52.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.057+0000 7f4c6b7fe700 1 -- 172.21.15.38:0/305581774 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943808994 0 0) 0x7f4c6c090f70 con 0x7f4c6c08bce0 2020-03-19T23:10:52.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6bfff700 1 --2- 172.21.15.38:0/305581774 >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4c5c002f30 tx=0x7f4c5c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6bfff700 1 --2- 172.21.15.38:0/305581774 >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4c5c002f30 tx=0x7f4c5c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6bfff700 1 --2- 172.21.15.38:0/305581774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f4c5c002f30 tx=0x7f4c5c007a40).ready entity=mon.0 client_cookie=b799258ef07fd1da server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6bfff700 1 --2- 172.21.15.38:0/305581774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7f4c5c002f30 tx=0x7f4c5c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6b7fe700 1 -- 172.21.15.38:0/305581774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4c5c008e80 con 0x7f4c6c08c7d0 2020-03-19T23:10:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6bfff700 1 --2- 172.21.15.38:0/305581774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7f4c5c002f30 tx=0x7f4c5c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6b7fe700 1 -- 172.21.15.38:0/305581774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c0113f0 con 0x7f4c6c08c7d0 2020-03-19T23:10:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6bfff700 1 --2- 172.21.15.38:0/305581774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 secure :-1 s=THROTTLE_DONE pgs=22 cs=0 l=1 rx=0x7f4c5c002f30 tx=0x7f4c5c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c6b7fe700 1 -- 172.21.15.38:0/305581774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4c5c0116d0 con 0x7f4c6c08c7d0 2020-03-19T23:10:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c72a3f700 1 -- 172.21.15.38:0/305581774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08c7d0 msgr2=0x7f4c6c087cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c72a3f700 1 --2- 172.21.15.38:0/305581774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f4c5c002f30 tx=0x7f4c5c007a40).stop 2020-03-19T23:10:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c72a3f700 1 -- 172.21.15.38:0/305581774 shutdown_connections 2020-03-19T23:10:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c72a3f700 1 --2- 172.21.15.38:0/305581774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08c7d0 0x7f4c6c087cb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c72a3f700 1 -- 172.21.15.38:0/305581774 >> 172.21.15.38:0/305581774 conn(0x7f4c6c0951d0 msgr2=0x7f4c6c091250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c72a3f700 1 -- 172.21.15.38:0/305581774 shutdown_connections 2020-03-19T23:10:52.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.058+0000 7f4c72a3f700 1 -- 172.21.15.38:0/305581774 wait complete. 2020-03-19T23:10:52.067 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.059+0000 7f4c72a3f700 1 Processor -- start 2020-03-19T23:10:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.059+0000 7f4c72a3f700 1 -- start start 2020-03-19T23:10:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c72a3f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08bce0 0x7f4c6c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c72a3f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c072630 con 0x7f4c6c08c7d0 2020-03-19T23:10:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c72a3f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c073dd0 con 0x7f4c6c08bce0 2020-03-19T23:10:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c70fdc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08bce0 0x7f4c6c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c6bfff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4c6c08c7d0 0x7f4c6c070a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43640/0 (socket says 172.21.15.38:43640) 2020-03-19T23:10:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c6bfff700 1 -- 172.21.15.38:0/620599547 learned_addr learned my addr 172.21.15.38:0/620599547 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c70fdc700 1 -- 172.21.15.38:0/620599547 >> v1:172.21.15.38:6789/0 conn(0x7f4c6c08c7d0 legacy=0x7f4c6c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c70fdc700 1 -- 172.21.15.38:0/620599547 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c072d90 con 0x7f4c6c08bce0 2020-03-19T23:10:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147836964 0 0) 0x7f4c6c072630 con 0x7f4c6c08c7d0 2020-03-19T23:10:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.060+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> v2:172.21.15.38:3300/0 conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:52.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4c54005b20 con 0x7f4c6c08bce0 2020-03-19T23:10:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c072390 con 0x7f4c6c08bce0 2020-03-19T23:10:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54005c80 con 0x7f4c6c08bce0 2020-03-19T23:10:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4c540060c0 con 0x7f4c6c08bce0 2020-03-19T23:10:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c071670 con 0x7f4c6c08bce0 2020-03-19T23:10:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f4c5400a6e0 con 0x7f4c6c08bce0 2020-03-19T23:10:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.061+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f4c540062f0 con 0x7f4c6c08bce0 2020-03-19T23:10:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.062+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c056db0 con 0x7f4c6c08bce0 2020-03-19T23:10:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.064+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.064+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f4c540090b0 con 0x7f4c6c08bce0 2020-03-19T23:10:52.156 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.154+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f4403c1d-0ebc-4a48-9e62-aa667c0c7f36", "id": 2} v 0) v1 -- 0x7f4c6c08bac0 con 0x7f4c6c08bce0 2020-03-19T23:10:52.159 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.157+0000 7f4c70fdc700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=THROTTLE_DONE pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.159 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.157+0000 7f4c697fa700 1 -- 172.21.15.38:0/620599547 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4403c1d-0ebc-4a48-9e62-aa667c0c7f36", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c5400f140 con 0x7f4c6c08bce0 2020-03-19T23:10:52.159 INFO:teuthology.orchestra.run.smithi038.stdout:2 2020-03-19T23:10:52.160 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.158+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 msgr2=0x7f4c6c0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.160 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.158+0000 7f4c72a3f700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7f4c54000b80 tx=0x7f4c54010950).stop 2020-03-19T23:10:52.160 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.158+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 shutdown_connections 2020-03-19T23:10:52.160 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.158+0000 7f4c72a3f700 1 --2- 172.21.15.38:0/620599547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c6c08bce0 0x7f4c6c0705d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:52.161 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.158+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 >> 172.21.15.38:0/620599547 conn(0x7f4c6c0951d0 msgr2=0x7f4c6c062870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:52.161 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.158+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 shutdown_connections 2020-03-19T23:10:52.161 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:52.158+0000 7f4c72a3f700 1 -- 172.21.15.38:0/620599547 wait complete. 2020-03-19T23:10:52.170 INFO:tasks.ceph.osd.0:Restarting daemon 2020-03-19T23:10:52.170 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-03-19T23:10:52.172 INFO:tasks.ceph.osd.0:Started 2020-03-19T23:10:52.172 INFO:tasks.ceph.osd.1:Restarting daemon 2020-03-19T23:10:52.172 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-03-19T23:10:52.174 INFO:tasks.ceph.osd.1:Started 2020-03-19T23:10:52.174 INFO:tasks.ceph.osd.2:Restarting daemon 2020-03-19T23:10:52.175 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-03-19T23:10:52.177 INFO:tasks.ceph.osd.2:Started 2020-03-19T23:10:52.177 INFO:tasks.ceph:Waiting for OSDs to come up 2020-03-19T23:10:52.479 INFO:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-03-19T23:10:52.579 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69f1a30700 1 Processor -- start 2020-03-19T23:10:52.579 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69f1a30700 1 -- start start 2020-03-19T23:10:52.579 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69f1a30700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec07fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:52.580 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69f1a30700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec05e8d0 con 0x7f69ec08af10 2020-03-19T23:10:52.580 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69f1a30700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec05ede0 con 0x7f69ec08ba00 2020-03-19T23:10:52.580 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69e2ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec07fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:52.580 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69eb7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f69ec08af10 0x7f69ec08b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43654/0 (socket says 172.21.15.38:43654) 2020-03-19T23:10:52.580 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.575+0000 7f69eb7fe700 1 -- 172.21.15.38:0/554490307 learned_addr learned my addr 172.21.15.38:0/554490307 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:52.580 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 -- 172.21.15.38:0/554490307 >> v1:172.21.15.38:6789/0 conn(0x7f69ec08af10 legacy=0x7f69ec08b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.580 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 -- 172.21.15.38:0/554490307 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec090af0 con 0x7f69ec08ba00 2020-03-19T23:10:52.581 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69eaffd700 1 -- 172.21.15.38:0/554490307 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682052453 0 0) 0x7f69ec05e8d0 con 0x7f69ec08af10 2020-03-19T23:10:52.581 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/554490307 >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec07fc30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f69ec08d8a0 tx=0x7f69d40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.581 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/554490307 >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec07fc30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f69ec08d8a0 tx=0x7f69d40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.581 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/554490307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec07fc30 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69d40088e0).ready entity=mon.0 client_cookie=14437353df83a8b8 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:52.581 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/554490307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec07fc30 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69d40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.581 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69eaffd700 1 -- 172.21.15.38:0/554490307 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f69d40113f0 con 0x7f69ec08ba00 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/554490307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec07fc30 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69d40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69eaffd700 1 -- 172.21.15.38:0/554490307 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d4011550 con 0x7f69ec08ba00 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/554490307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec07fc30 secure :-1 s=THROTTLE_DONE pgs=32 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69d40088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69eaffd700 1 -- 172.21.15.38:0/554490307 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f69d4011830 con 0x7f69ec08ba00 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69f1a30700 1 -- 172.21.15.38:0/554490307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 msgr2=0x7f69ec07fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69f1a30700 1 --2- 172.21.15.38:0/554490307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec07fc30 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69d40088e0).stop 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69f1a30700 1 -- 172.21.15.38:0/554490307 shutdown_connections 2020-03-19T23:10:52.582 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69f1a30700 1 --2- 172.21.15.38:0/554490307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec07fc30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:52.583 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69f1a30700 1 -- 172.21.15.38:0/554490307 >> 172.21.15.38:0/554490307 conn(0x7f69ec0955a0 msgr2=0x7f69ec091460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:52.583 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69f1a30700 1 -- 172.21.15.38:0/554490307 shutdown_connections 2020-03-19T23:10:52.583 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.576+0000 7f69f1a30700 1 -- 172.21.15.38:0/554490307 wait complete. 2020-03-19T23:10:52.583 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.577+0000 7f69f1a30700 1 Processor -- start 2020-03-19T23:10:52.585 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69f1a30700 1 -- start start 2020-03-19T23:10:52.585 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69f1a30700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec066fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:52.586 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69f1a30700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec060d10 con 0x7f69ec08af10 2020-03-19T23:10:52.586 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69f1a30700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec08a5d0 con 0x7f69ec08ba00 2020-03-19T23:10:52.586 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69e2ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec066fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:52.586 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69eb7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f69ec08af10 0x7f69ec066890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43658/0 (socket says 172.21.15.38:43658) 2020-03-19T23:10:52.586 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69eb7fe700 1 -- 172.21.15.38:0/637225578 learned_addr learned my addr 172.21.15.38:0/637225578 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:52.586 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69e2ffd700 1 -- 172.21.15.38:0/637225578 >> v1:172.21.15.38:6789/0 conn(0x7f69ec08af10 legacy=0x7f69ec066890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.586 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979013978 0 0) 0x7f69ec060d10 con 0x7f69ec08af10 2020-03-19T23:10:52.587 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69e2ffd700 1 -- 172.21.15.38:0/637225578 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec08a330 con 0x7f69ec08ba00 2020-03-19T23:10:52.587 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.587 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> v2:172.21.15.38:3300/0 conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.587 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.578+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:52.587 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.587 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f69d400f7c0 con 0x7f69ec08ba00 2020-03-19T23:10:52.587 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.588 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.588 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d40056c0 con 0x7f69ec08ba00 2020-03-19T23:10:52.588 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec061640 con 0x7f69ec08ba00 2020-03-19T23:10:52.588 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f69d4005960 con 0x7f69ec08ba00 2020-03-19T23:10:52.588 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec08ac30 con 0x7f69ec08ba00 2020-03-19T23:10:52.588 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.588 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 360+0+0 (secure 0 0 0) 0x7f69d4015250 con 0x7f69ec08ba00 2020-03-19T23:10:52.589 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.589 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.579+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f69d4015580 con 0x7f69ec08ba00 2020-03-19T23:10:52.589 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.580+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec0675c0 con 0x7f69ec08ba00 2020-03-19T23:10:52.589 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.582+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.589 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.582+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+68559 (secure 0 0 0) 0x7f69d4015b10 con 0x7f69ec08ba00 2020-03-19T23:10:52.658 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.656+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69ec056db0 con 0x7f69ec08ba00 2020-03-19T23:10:52.659 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.657+0000 7f69e2ffd700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=THROTTLE_DONE pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:52.659 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.657+0000 7f69e8ff9700 1 -- 172.21.15.38:0/637225578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) v1 ==== 74+0+3218 (secure 0 0 0) 0x7f69d4015d30 con 0x7f69ec08ba00 2020-03-19T23:10:52.659 INFO:teuthology.misc.health.smithi038.stdout: 2020-03-19T23:10:52.659 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":5,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:10:52.156170+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:10:52.660 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.658+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 msgr2=0x7f69ec066fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:52.660 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.658+0000 7f69f1a30700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7f69ec08d8a0 tx=0x7f69ec08e400).stop 2020-03-19T23:10:52.661 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.658+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 shutdown_connections 2020-03-19T23:10:52.661 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.658+0000 7f69f1a30700 1 --2- 172.21.15.38:0/637225578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69ec08ba00 0x7f69ec066fa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:52.661 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.658+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 >> 172.21.15.38:0/637225578 conn(0x7f69ec0955a0 msgr2=0x7f69ec07fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:52.661 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.658+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 shutdown_connections 2020-03-19T23:10:52.661 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:52.658+0000 7f69f1a30700 1 -- 172.21.15.38:0/637225578 wait complete. 2020-03-19T23:10:52.669 DEBUG:teuthology.misc:0 of 3 OSDs are up 2020-03-19T23:10:52.776 INFO:tasks.ceph.osd.1.smithi038.stderr:2020-03-19T23:10:52.774+0000 7f88edb0cec0 -1 Falling back to public interface 2020-03-19T23:10:52.785 INFO:tasks.ceph.osd.0.smithi038.stderr:2020-03-19T23:10:52.783+0000 7f5e63774ec0 -1 Falling back to public interface 2020-03-19T23:10:52.796 INFO:tasks.ceph.osd.2.smithi038.stderr:2020-03-19T23:10:52.794+0000 7efc00534ec0 -1 Falling back to public interface 2020-03-19T23:10:53.334 INFO:tasks.ceph.osd.1.smithi038.stderr:2020-03-19T23:10:53.332+0000 7f88edb0cec0 -1 osd.1 0 log_to_monitors {default=true} 2020-03-19T23:10:53.341 INFO:tasks.ceph.osd.0.smithi038.stderr:2020-03-19T23:10:53.339+0000 7f5e63774ec0 -1 osd.0 0 log_to_monitors {default=true} 2020-03-19T23:10:53.349 INFO:tasks.ceph.osd.2.smithi038.stderr:2020-03-19T23:10:53.347+0000 7efc00534ec0 -1 osd.2 0 log_to_monitors {default=true} 2020-03-19T23:10:55.218 INFO:tasks.ceph.osd.0.smithi038.stderr:2020-03-19T23:10:55.216+0000 7f5e48738700 -1 osd.0 0 waiting for initial osdmap 2020-03-19T23:10:55.218 INFO:tasks.ceph.osd.1.smithi038.stderr:2020-03-19T23:10:55.217+0000 7f88d2ad0700 -1 osd.1 0 waiting for initial osdmap 2020-03-19T23:10:55.218 INFO:tasks.ceph.osd.2.smithi038.stderr:2020-03-19T23:10:55.217+0000 7efbe54f8700 -1 osd.2 0 waiting for initial osdmap 2020-03-19T23:10:58.971 INFO:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-03-19T23:10:59.087 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.079+0000 7f2e5c4fc700 1 Processor -- start 2020-03-19T23:10:59.087 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.079+0000 7f2e5c4fc700 1 -- start start 2020-03-19T23:10:59.088 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.079+0000 7f2e5c4fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e5408b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:59.088 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5c4fc700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5405e8d0 con 0x7f2e5408b750 2020-03-19T23:10:59.088 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5c4fc700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5405ede0 con 0x7f2e5408af10 2020-03-19T23:10:59.088 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e5408b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:59.089 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5a298700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2e5408b750 0x7f2e5408bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43758/0 (socket says 172.21.15.38:43758) 2020-03-19T23:10:59.089 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e5408b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60618/0 (socket says 172.21.15.38:60618) 2020-03-19T23:10:59.089 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5a298700 1 -- 172.21.15.38:0/3509529748 learned_addr learned my addr 172.21.15.38:0/3509529748 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:59.089 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 -- 172.21.15.38:0/3509529748 >> v1:172.21.15.38:6789/0 conn(0x7f2e5408b750 legacy=0x7f2e5408bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.090 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 -- 172.21.15.38:0/3509529748 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e54090af0 con 0x7f2e5408af10 2020-03-19T23:10:59.090 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3509529748 >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e5408b2f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.090 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3509529748 >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e5408b2f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.090 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3509529748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e5408b2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).ready entity=mon.0 client_cookie=5d38bfebfc9ea470 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:59.091 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3509529748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e5408b2f0 secure :-1 s=THROTTLE_DONE pgs=51 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.091 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e59a97700 1 -- 172.21.15.38:0/3509529748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2e500113f0 con 0x7f2e5408af10 2020-03-19T23:10:59.091 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3509529748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e5408b2f0 secure :-1 s=THROTTLE_DONE pgs=51 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.091 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e59a97700 1 -- 172.21.15.38:0/3509529748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50011550 con 0x7f2e5408af10 2020-03-19T23:10:59.092 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3509529748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e5408b2f0 secure :-1 s=THROTTLE_DONE pgs=51 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.092 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e59a97700 1 -- 172.21.15.38:0/3509529748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2e50011830 con 0x7f2e5408af10 2020-03-19T23:10:59.092 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3509529748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 msgr2=0x7f2e5408b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.092 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.080+0000 7f2e5c4fc700 1 --2- 172.21.15.38:0/3509529748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e5408b2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).stop 2020-03-19T23:10:59.093 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.081+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3509529748 shutdown_connections 2020-03-19T23:10:59.093 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.081+0000 7f2e5c4fc700 1 --2- 172.21.15.38:0/3509529748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e5408b2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:59.093 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.081+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3509529748 >> 172.21.15.38:0/3509529748 conn(0x7f2e540955a0 msgr2=0x7f2e54091460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:59.093 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.081+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3509529748 shutdown_connections 2020-03-19T23:10:59.094 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.081+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3509529748 wait complete. 2020-03-19T23:10:59.094 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5c4fc700 1 Processor -- start 2020-03-19T23:10:59.094 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5c4fc700 1 -- start start 2020-03-19T23:10:59.094 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5c4fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e54083640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:59.095 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5c4fc700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e540642e0 con 0x7f2e5408b750 2020-03-19T23:10:59.095 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5c4fc700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e540633c0 con 0x7f2e5408af10 2020-03-19T23:10:59.095 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5aa99700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e54083640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:59.095 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5a298700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2e5408b750 0x7f2e54061ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43764/0 (socket says 172.21.15.38:43764) 2020-03-19T23:10:59.096 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.082+0000 7f2e5a298700 1 -- 172.21.15.38:0/3128801722 learned_addr learned my addr 172.21.15.38:0/3128801722 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:59.096 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 -- 172.21.15.38:0/3128801722 >> v1:172.21.15.38:6789/0 conn(0x7f2e5408b750 legacy=0x7f2e54061ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.096 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 -- 172.21.15.38:0/3128801722 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e54083370 con 0x7f2e5408af10 2020-03-19T23:10:59.096 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987319884 0 0) 0x7f2e540642e0 con 0x7f2e5408b750 2020-03-19T23:10:59.097 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.097 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> v2:172.21.15.38:3300/0 conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.097 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:59.097 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.098 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2e5000f7c0 con 0x7f2e5408af10 2020-03-19T23:10:59.098 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.098 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.098 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50007150 con 0x7f2e5408af10 2020-03-19T23:10:59.099 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2e500073f0 con 0x7f2e5408af10 2020-03-19T23:10:59.099 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e54093290 con 0x7f2e5408af10 2020-03-19T23:10:59.099 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e54067650 con 0x7f2e5408af10 2020-03-19T23:10:59.099 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.083+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.100 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.100 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f2e500056c0 con 0x7f2e5408af10 2020-03-19T23:10:59.100 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e4b7fe700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 0x7f2e3c02a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:59.100 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2230+0+0 (secure 0 0 0) 0x7f2e500348f0 con 0x7f2e5408af10 2020-03-19T23:10:59.100 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e5a298700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 0x7f2e3c02a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:59.101 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e5a298700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 0x7f2e3c02a770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2e44005500 tx=0x7f2e440016a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.101 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e540640e0 con 0x7f2e5408af10 2020-03-19T23:10:59.101 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e5a298700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 0x7f2e3c02a770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2e44005500 tx=0x7f2e440016a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.114 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.084+0000 7f2e5a298700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 0x7f2e3c02a770 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f2e44005500 tx=0x7f2e440016a0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:59.115 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.088+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.115 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.088+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f2e50006070 con 0x7f2e5408af10 2020-03-19T23:10:59.237 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.236+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e54056db0 con 0x7f2e5408af10 2020-03-19T23:10:59.238 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.237+0000 7f2e5aa99700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=THROTTLE_DONE pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.238 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.237+0000 7f2e4b7fe700 1 -- 172.21.15.38:0/3128801722 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+4572 (secure 0 0 0) 0x7f2e50006070 con 0x7f2e5408af10 2020-03-19T23:10:59.238 INFO:teuthology.misc.health.smithi038.stdout: 2020-03-19T23:10:59.239 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":9,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:10:57.221362+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:10:59.240 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 msgr2=0x7f2e3c02a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.240 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 0x7f2e3c02a770 secure :-1 s=READY pgs=14 cs=0 l=1 rx=0x7f2e44005500 tx=0x7f2e440016a0).stop 2020-03-19T23:10:59.240 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 msgr2=0x7f2e54083640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.240 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f2e50002f30 tx=0x7f2e50007a40).stop 2020-03-19T23:10:59.240 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 shutdown_connections 2020-03-19T23:10:59.240 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e5408af10 0x7f2e54083640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:59.241 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 --2- 172.21.15.38:0/3128801722 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2e3c0282b0 0x7f2e3c02a770 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:59.241 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 >> 172.21.15.38:0/3128801722 conn(0x7f2e540955a0 msgr2=0x7f2e5408a220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:59.241 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 shutdown_connections 2020-03-19T23:10:59.241 INFO:teuthology.misc.health.smithi038.stderr:2020-03-19T23:10:59.238+0000 7f2e5c4fc700 1 -- 172.21.15.38:0/3128801722 wait complete. 2020-03-19T23:10:59.248 DEBUG:teuthology.misc:3 of 3 OSDs are up 2020-03-19T23:10:59.249 INFO:tasks.ceph:Creating RBD pool 2020-03-19T23:10:59.249 INFO:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd pool create rbd 8 2020-03-19T23:10:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.360+0000 7f2aac519700 1 Processor -- start 2020-03-19T23:10:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.360+0000 7f2aac519700 1 -- start start 2020-03-19T23:10:59.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aac519700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f010 0x7f2aa408f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:10:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aac519700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa40902c0 con 0x7f2aa408f850 2020-03-19T23:10:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aac519700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa40904f0 con 0x7f2aa408f010 2020-03-19T23:10:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aaaab6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f010 0x7f2aa408f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aaa2b5700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2aa408f850 0x7f2aa4136ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43768/0 (socket says 172.21.15.38:43768) 2020-03-19T23:10:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aaa2b5700 1 -- 172.21.15.38:0/2912185472 learned_addr learned my addr 172.21.15.38:0/2912185472 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:59.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aaaab6700 1 -- 172.21.15.38:0/2912185472 >> v1:172.21.15.38:6789/0 conn(0x7f2aa408f850 legacy=0x7f2aa4136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aa9ab4700 1 -- 172.21.15.38:0/2912185472 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416708160 0 0) 0x7f2aa40902c0 con 0x7f2aa408f850 2020-03-19T23:10:59.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aaaab6700 1 -- 172.21.15.38:0/2912185472 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40954e0 con 0x7f2aa408f010 2020-03-19T23:10:59.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.361+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2912185472 >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f010 0x7f2aa408f3f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f2aa0002f30 tx=0x7f2aa0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2912185472 >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f010 0x7f2aa408f3f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f2aa0002f30 tx=0x7f2aa0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2912185472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f010 0x7f2aa408f3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f2aa0002f30 tx=0x7f2aa0007a40).ready entity=mon.0 client_cookie=e268e5d99ff477bf server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:59.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2912185472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f010 0x7f2aa408f3f0 secure :-1 s=THROTTLE_DONE pgs=55 cs=0 l=1 rx=0x7f2aa0002f30 tx=0x7f2aa0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aa9ab4700 1 -- 172.21.15.38:0/2912185472 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2aa0008e80 con 0x7f2aa408f010 2020-03-19T23:10:59.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2912185472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f010 0x7f2aa408f3f0 secure :-1 s=THROTTLE_DONE pgs=55 cs=0 l=1 rx=0x7f2aa0002f30 tx=0x7f2aa0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aa9ab4700 1 -- 172.21.15.38:0/2912185472 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa00113f0 con 0x7f2aa408f010 2020-03-19T23:10:59.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2912185472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f010 0x7f2aa408f3f0 secure :-1 s=THROTTLE_DONE pgs=55 cs=0 l=1 rx=0x7f2aa0002f30 tx=0x7f2aa0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aa9ab4700 1 -- 172.21.15.38:0/2912185472 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2aa00116d0 con 0x7f2aa408f010 2020-03-19T23:10:59.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aac519700 1 -- 172.21.15.38:0/2912185472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f010 msgr2=0x7f2aa408f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aac519700 1 --2- 172.21.15.38:0/2912185472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f010 0x7f2aa408f3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f2aa0002f30 tx=0x7f2aa0007a40).stop 2020-03-19T23:10:59.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aac519700 1 -- 172.21.15.38:0/2912185472 shutdown_connections 2020-03-19T23:10:59.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aac519700 1 --2- 172.21.15.38:0/2912185472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f010 0x7f2aa408f3f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:10:59.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aac519700 1 -- 172.21.15.38:0/2912185472 >> 172.21.15.38:0/2912185472 conn(0x7f2aa4095090 msgr2=0x7f2aa4091110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:10:59.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aac519700 1 -- 172.21.15.38:0/2912185472 shutdown_connections 2020-03-19T23:10:59.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.362+0000 7f2aac519700 1 -- 172.21.15.38:0/2912185472 wait complete. 2020-03-19T23:10:59.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.363+0000 7f2aac519700 1 Processor -- start 2020-03-19T23:10:59.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.363+0000 7f2aac519700 1 -- start start 2020-03-19T23:10:59.373 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aac519700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f850 0x7f2aa4073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:59.373 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aac519700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa4074d70 con 0x7f2aa408f010 2020-03-19T23:10:59.373 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aac519700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa40757e0 con 0x7f2aa408f850 2020-03-19T23:10:59.373 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaaab6700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2aa408f010 0x7f2aa4072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43772/0 (socket says 172.21.15.38:43772) 2020-03-19T23:10:59.373 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaaab6700 1 -- 172.21.15.38:0/2805046790 learned_addr learned my addr 172.21.15.38:0/2805046790 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:10:59.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f850 0x7f2aa4073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:59.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152308221 0 0) 0x7f2aa4074d70 con 0x7f2aa408f010 2020-03-19T23:10:59.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a8c003230 con 0x7f2aa408f010 2020-03-19T23:10:59.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 -- 172.21.15.38:0/2805046790 >> v1:172.21.15.38:6789/0 conn(0x7f2aa408f010 legacy=0x7f2aa4072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:10:59.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 -- 172.21.15.38:0/2805046790 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4075440 con 0x7f2aa408f850 2020-03-19T23:10:59.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> v2:172.21.15.38:3300/0 conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.375 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:59.375 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.375 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2a98005b20 con 0x7f2aa408f850 2020-03-19T23:10:59.375 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.375 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.375 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a98005c80 con 0x7f2aa408f850 2020-03-19T23:10:59.375 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2a980060c0 con 0x7f2aa408f850 2020-03-19T23:10:59.376 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa40938a0 con 0x7f2aa408f850 2020-03-19T23:10:59.376 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.364+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa407d910 con 0x7f2aa408f850 2020-03-19T23:10:59.376 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.365+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.376 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.365+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f2a9800a320 con 0x7f2aa408f850 2020-03-19T23:10:59.376 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.365+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.376 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.365+0000 7f2a977fe700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 0x7f2a8c02ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:10:59.376 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.365+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2230+0+0 (secure 0 0 0) 0x7f2a98034070 con 0x7f2aa408f850 2020-03-19T23:10:59.377 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.365+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 0x7f2a8c02ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:10:59.377 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.365+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 0x7f2a8c02ac80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2aa0000b80 tx=0x7f2aa0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.377 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.366+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 0x7f2a8c02ac80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2aa0000b80 tx=0x7f2aa0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.377 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.366+0000 7f2aaaab6700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 0x7f2a8c02ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rx=0x7f2aa0000b80 tx=0x7f2aa0007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:10:59.377 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.366+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa4056db0 con 0x7f2aa408f850 2020-03-19T23:10:59.377 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.369+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:10:59.377 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.369+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f2a98006220 con 0x7f2aa408f850 2020-03-19T23:10:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:10:59.515+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f2aa407e050 con 0x7f2aa408f850 2020-03-19T23:11:00.242 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.241+0000 7f2aaa2b5700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=THROTTLE_DONE pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.242 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.241+0000 7f2a977fe700 1 -- 172.21.15.38:0/2805046790 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) v1 ==== 109+0+0 (secure 0 0 0) 0x7f2a98011d00 con 0x7f2aa408f850 2020-03-19T23:11:00.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 msgr2=0x7f2a8c02ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:00.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 0x7f2a8c02ac80 secure :-1 s=READY pgs=15 cs=0 l=1 rx=0x7f2aa0000b80 tx=0x7f2aa0007a40).stop 2020-03-19T23:11:00.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 msgr2=0x7f2aa4073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:00.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7f2a98000b80 tx=0x7f2a98010860).stop 2020-03-19T23:11:00.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 shutdown_connections 2020-03-19T23:11:00.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2a8c0287c0 0x7f2a8c02ac80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:00.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 --2- 172.21.15.38:0/2805046790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa408f850 0x7f2aa4073150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:00.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 >> 172.21.15.38:0/2805046790 conn(0x7f2aa4095090 msgr2=0x7f2aa4061310 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:00.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 shutdown_connections 2020-03-19T23:11:00.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.245+0000 7f2aac519700 1 -- 172.21.15.38:0/2805046790 wait complete. 2020-03-19T23:11:00.248 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'rbd' created 2020-03-19T23:11:00.259 INFO:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-03-19T23:11:00.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.378+0000 7f95e5f33700 1 Processor -- start 2020-03-19T23:11:00.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.378+0000 7f95e5f33700 1 -- start start 2020-03-19T23:11:00.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95e5f33700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0137d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:00.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95e5f33700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00902a0 con 0x7f95e0091890 2020-03-19T23:11:00.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95e5f33700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e00600e0 con 0x7f95e008f790 2020-03-19T23:11:00.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0137d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:00.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95dffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f95e0091890 0x7f95e008f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43778/0 (socket says 172.21.15.38:43778) 2020-03-19T23:11:00.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95dffff700 1 -- 172.21.15.38:0/2089946999 learned_addr learned my addr 172.21.15.38:0/2089946999 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:00.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 -- 172.21.15.38:0/2089946999 >> v1:172.21.15.38:6789/0 conn(0x7f95e0091890 legacy=0x7f95e008f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:00.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 -- 172.21.15.38:0/2089946999 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e00954e0 con 0x7f95e008f790 2020-03-19T23:11:00.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 --2- 172.21.15.38:0/2089946999 >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0137d60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f95c8002f30 tx=0x7f95c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 --2- 172.21.15.38:0/2089946999 >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0137d60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f95c8002f30 tx=0x7f95c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.384 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 --2- 172.21.15.38:0/2089946999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0137d60 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f95c8002f30 tx=0x7f95c8007a40).ready entity=mon.0 client_cookie=7f31ac4b2f8941b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:00.384 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 --2- 172.21.15.38:0/2089946999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0137d60 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f95c8002f30 tx=0x7f95c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.384 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95df7fe700 1 -- 172.21.15.38:0/2089946999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f95c8008e80 con 0x7f95e008f790 2020-03-19T23:11:00.384 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 --2- 172.21.15.38:0/2089946999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0137d60 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f95c8002f30 tx=0x7f95c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.384 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95df7fe700 1 -- 172.21.15.38:0/2089946999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c80113f0 con 0x7f95e008f790 2020-03-19T23:11:00.384 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95d7fff700 1 --2- 172.21.15.38:0/2089946999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0137d60 secure :-1 s=THROTTLE_DONE pgs=59 cs=0 l=1 rx=0x7f95c8002f30 tx=0x7f95c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.384 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95df7fe700 1 -- 172.21.15.38:0/2089946999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f95c8011710 con 0x7f95e008f790 2020-03-19T23:11:00.385 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95e5f33700 1 -- 172.21.15.38:0/2089946999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 msgr2=0x7f95e0137d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:00.385 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.379+0000 7f95e5f33700 1 --2- 172.21.15.38:0/2089946999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0137d60 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f95c8002f30 tx=0x7f95c8007a40).stop 2020-03-19T23:11:00.385 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.380+0000 7f95e5f33700 1 -- 172.21.15.38:0/2089946999 shutdown_connections 2020-03-19T23:11:00.385 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.380+0000 7f95e5f33700 1 --2- 172.21.15.38:0/2089946999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0137d60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:00.385 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.380+0000 7f95e5f33700 1 -- 172.21.15.38:0/2089946999 >> 172.21.15.38:0/2089946999 conn(0x7f95e0095100 msgr2=0x7f95e0091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:00.385 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.380+0000 7f95e5f33700 1 -- 172.21.15.38:0/2089946999 shutdown_connections 2020-03-19T23:11:00.385 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.380+0000 7f95e5f33700 1 -- 172.21.15.38:0/2089946999 wait complete. 2020-03-19T23:11:00.386 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95e5f33700 1 Processor -- start 2020-03-19T23:11:00.386 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95e5f33700 1 -- start start 2020-03-19T23:11:00.389 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95e5f33700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:00.389 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95e5f33700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e0075fa0 con 0x7f95e0091890 2020-03-19T23:11:00.389 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95e5f33700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e00761d0 con 0x7f95e008f790 2020-03-19T23:11:00.389 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95dffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:00.389 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95d7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f95e0091890 0x7f95e0072e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43780/0 (socket says 172.21.15.38:43780) 2020-03-19T23:11:00.389 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.381+0000 7f95d7fff700 1 -- 172.21.15.38:0/4281834511 learned_addr learned my addr 172.21.15.38:0/4281834511 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:00.390 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544821605 0 0) 0x7f95e0075fa0 con 0x7f95e0091890 2020-03-19T23:11:00.390 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4003230 con 0x7f95e0091890 2020-03-19T23:11:00.390 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 -- 172.21.15.38:0/4281834511 >> v1:172.21.15.38:6789/0 conn(0x7f95e0091890 legacy=0x7f95e0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:00.390 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 -- 172.21.15.38:0/4281834511 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0092fb0 con 0x7f95e008f790 2020-03-19T23:11:00.390 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.390 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> v2:172.21.15.38:3300/0 conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:00.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f95d0005b20 con 0x7f95e008f790 2020-03-19T23:11:00.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0011e90 con 0x7f95e008f790 2020-03-19T23:11:00.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e0073f10 con 0x7f95e008f790 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f95d0005e30 con 0x7f95e008f790 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.382+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e0073770 con 0x7f95e008f790 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f95d000a320 con 0x7f95e008f790 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95dd7fa700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 0x7f95c402aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2572+0+0 (secure 0 0 0) 0x7f95d0033f40 con 0x7f95e008f790 2020-03-19T23:11:00.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95d7fff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 0x7f95c402aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:00.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95d7fff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 0x7f95c402aca0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f95c8000b80 tx=0x7f95c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0069270 con 0x7f95e008f790 2020-03-19T23:11:00.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95d7fff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 0x7f95c402aca0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f95c8000b80 tx=0x7f95c8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.383+0000 7f95d7fff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 0x7f95c402aca0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f95c8000b80 tx=0x7f95c8007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:00.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.387+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:00.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.387+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f95d0006190 con 0x7f95e008f790 2020-03-19T23:11:00.536 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:00.535+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f95e0075de0 con 0x7f95e008f790 2020-03-19T23:11:01.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.245+0000 7f95dffff700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=THROTTLE_DONE pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.245+0000 7f95dd7fa700 1 -- 172.21.15.38:0/4281834511 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) v1 ==== 173+0+0 (secure 0 0 0) 0x7f95d0011c90 con 0x7f95e008f790 2020-03-19T23:11:01.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.251+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 msgr2=0x7f95c402aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.251+0000 7f95e5f33700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 0x7f95c402aca0 secure :-1 s=READY pgs=16 cs=0 l=1 rx=0x7f95c8000b80 tx=0x7f95c8007a40).stop 2020-03-19T23:11:01.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.251+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 msgr2=0x7f95e0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.251+0000 7f95e5f33700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7f95e006aca0 tx=0x7f95d00107f0).stop 2020-03-19T23:11:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.253+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 shutdown_connections 2020-03-19T23:11:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.253+0000 7f95e5f33700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95e008f790 0x7f95e0074a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.253+0000 7f95e5f33700 1 --2- 172.21.15.38:0/4281834511 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f95c40287e0 0x7f95c402aca0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.253+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 >> 172.21.15.38:0/4281834511 conn(0x7f95e0095100 msgr2=0x7f95e00632b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.253+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 shutdown_connections 2020-03-19T23:11:01.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.253+0000 7f95e5f33700 1 -- 172.21.15.38:0/4281834511 wait complete. 2020-03-19T23:11:01.255 INFO:teuthology.orchestra.run.smithi038.stderr:enabled application 'rbd' on pool 'rbd' 2020-03-19T23:11:01.267 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-03-19T23:11:01.267 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:01.386 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.385+0000 7f1dc4879700 1 Processor -- start 2020-03-19T23:11:01.387 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.385+0000 7f1dc4879700 1 -- start start 2020-03-19T23:11:01.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dc4879700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc0127f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:01.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dc4879700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc0090af0 con 0x7f1dc0065d30 2020-03-19T23:11:01.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dc4879700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc0066820 con 0x7f1dc0125ba0 2020-03-19T23:11:01.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dbdd9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc0127f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dbe59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1dc0065d30 0x7f1dc0066110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43786/0 (socket says 172.21.15.38:43786) 2020-03-19T23:11:01.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dbe59c700 1 -- 172.21.15.38:0/2566551630 learned_addr learned my addr 172.21.15.38:0/2566551630 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:01.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dbdd9b700 1 -- 172.21.15.38:0/2566551630 >> v1:172.21.15.38:6789/0 conn(0x7f1dc0065d30 legacy=0x7f1dc0066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dbd59a700 1 -- 172.21.15.38:0/2566551630 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882456080 0 0) 0x7f1dc0090af0 con 0x7f1dc0065d30 2020-03-19T23:11:01.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.386+0000 7f1dbdd9b700 1 -- 172.21.15.38:0/2566551630 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc008eec0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/2566551630 >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc0127f90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/2566551630 >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc0127f90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/2566551630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc0127f90 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).ready entity=mon.0 client_cookie=f7cfc36973114953 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/2566551630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc0127f90 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbd59a700 1 -- 172.21.15.38:0/2566551630 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1db0011cd0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/2566551630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc0127f90 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbd59a700 1 -- 172.21.15.38:0/2566551630 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db0011e30 con 0x7f1dc0125ba0 2020-03-19T23:11:01.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/2566551630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc0127f90 secure :-1 s=THROTTLE_DONE pgs=63 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dc4879700 1 -- 172.21.15.38:0/2566551630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 msgr2=0x7f1dc0127f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dbd59a700 1 -- 172.21.15.38:0/2566551630 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1db000a290 con 0x7f1dc0125ba0 2020-03-19T23:11:01.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dc4879700 1 --2- 172.21.15.38:0/2566551630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc0127f90 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).stop 2020-03-19T23:11:01.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dc4879700 1 -- 172.21.15.38:0/2566551630 shutdown_connections 2020-03-19T23:11:01.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dc4879700 1 --2- 172.21.15.38:0/2566551630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc0127f90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dc4879700 1 -- 172.21.15.38:0/2566551630 >> 172.21.15.38:0/2566551630 conn(0x7f1dc0091540 msgr2=0x7f1dc0091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:01.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dc4879700 1 -- 172.21.15.38:0/2566551630 shutdown_connections 2020-03-19T23:11:01.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.387+0000 7f1dc4879700 1 -- 172.21.15.38:0/2566551630 wait complete. 2020-03-19T23:11:01.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.388+0000 7f1dc4879700 1 Processor -- start 2020-03-19T23:11:01.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.388+0000 7f1dc4879700 1 -- start start 2020-03-19T23:11:01.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.388+0000 7f1dc4879700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc011f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:01.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.388+0000 7f1dc4879700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc011f5d0 con 0x7f1dc0065d30 2020-03-19T23:11:01.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.388+0000 7f1dc4879700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc011bc30 con 0x7f1dc0125ba0 2020-03-19T23:11:01.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc011f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbe59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1dc0065d30 0x7f1dc011c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43790/0 (socket says 172.21.15.38:43790) 2020-03-19T23:11:01.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbe59c700 1 -- 172.21.15.38:0/3828331512 learned_addr learned my addr 172.21.15.38:0/3828331512 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:01.397 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 -- 172.21.15.38:0/3828331512 >> v1:172.21.15.38:6789/0 conn(0x7f1dc0065d30 legacy=0x7f1dc011c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.397 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847069603 0 0) 0x7f1dc011f5d0 con 0x7f1dc0065d30 2020-03-19T23:11:01.397 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 -- 172.21.15.38:0/3828331512 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc011e9d0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.397 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.397 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> v2:172.21.15.38:3300/0 conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.398 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.399 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.399 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1db0005a40 con 0x7f1dc0125ba0 2020-03-19T23:11:01.399 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.399 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db0005ba0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.400 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc0125390 con 0x7f1dc0125ba0 2020-03-19T23:11:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1db0005e40 con 0x7f1dc0125ba0 2020-03-19T23:11:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.389+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc011f830 con 0x7f1dc0125ba0 2020-03-19T23:11:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f1db000a6e0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1daf7fe700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 0x7f1d9c02a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:01.401 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7f1db0034fe0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1dbe59c700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 0x7f1d9c02a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1dbe59c700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 0x7f1d9c02a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1db4005560 tx=0x7f1db40016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc011dfd0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1dbe59c700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 0x7f1d9c02a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1db4005560 tx=0x7f1db40016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.402 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.390+0000 7f1dbe59c700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 0x7f1d9c02a6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rx=0x7f1db4005560 tx=0x7f1db40016c0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.394+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.403 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.394+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f1db0006ad0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.542 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.541+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dc0056db0 con 0x7f1dc0125ba0 2020-03-19T23:11:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.542+0000 7f1dbdd9b700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=THROTTLE_DONE pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.543 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.542+0000 7f1daf7fe700 1 -- 172.21.15.38:0/3828331512 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+5957 (secure 0 0 0) 0x7f1db0006070 con 0x7f1dc0125ba0 2020-03-19T23:11:01.544 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:01.544 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":11,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:01.240666+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 msgr2=0x7f1d9c02a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.545 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 0x7f1d9c02a6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rx=0x7f1db4005560 tx=0x7f1db40016c0).stop 2020-03-19T23:11:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 msgr2=0x7f1dc011f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f1db00032b0 tx=0x7f1db0007dc0).stop 2020-03-19T23:11:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 shutdown_connections 2020-03-19T23:11:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028200 0x7f1d9c02a6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.546 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 --2- 172.21.15.38:0/3828331512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1dc0125ba0 0x7f1dc011f170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 >> 172.21.15.38:0/3828331512 conn(0x7f1dc0091540 msgr2=0x7f1dc0093500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 shutdown_connections 2020-03-19T23:11:01.547 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.543+0000 7f1dc4879700 1 -- 172.21.15.38:0/3828331512 wait complete. 2020-03-19T23:11:01.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:10:59.540840+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'11', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T23:11:01.555 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T23:11:01.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f206d78a700 1 Processor -- start 2020-03-19T23:11:01.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f206d78a700 1 -- start start 2020-03-19T23:11:01.687 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f206d78a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2068125840 0x7f2068127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:01.688 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f206d78a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20681228a0 con 0x7f20681224c0 2020-03-19T23:11:01.688 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f206d78a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2068092200 con 0x7f2068125840 2020-03-19T23:11:01.688 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f20667fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2068125840 0x7f2068127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.688 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f2066ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f20681224c0 0x7f2068122ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43796/0 (socket says 172.21.15.38:43796) 2020-03-19T23:11:01.689 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.671+0000 7f2066ffd700 1 -- 172.21.15.38:0/2043688683 learned_addr learned my addr 172.21.15.38:0/2043688683 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:01.689 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 -- 172.21.15.38:0/2043688683 >> v1:172.21.15.38:6789/0 conn(0x7f20681224c0 legacy=0x7f2068122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.689 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f2065ffb700 1 -- 172.21.15.38:0/2043688683 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216825303 0 0) 0x7f20600010d0 con 0x7f20681224c0 2020-03-19T23:11:01.689 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 -- 172.21.15.38:0/2043688683 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2068090b60 con 0x7f2068125840 2020-03-19T23:11:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 --2- 172.21.15.38:0/2043688683 >> v2:172.21.15.38:3300/0 conn(0x7f2068125840 0x7f2068127c30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f20500032b0 tx=0x7f2050007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 --2- 172.21.15.38:0/2043688683 >> v2:172.21.15.38:3300/0 conn(0x7f2068125840 0x7f2068127c30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f20500032b0 tx=0x7f2050007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 --2- 172.21.15.38:0/2043688683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2068125840 0x7f2068127c30 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f20500032b0 tx=0x7f2050007dc0).ready entity=mon.0 client_cookie=1a24c60edffb49ce server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 --2- 172.21.15.38:0/2043688683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2068125840 0x7f2068127c30 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7f20500032b0 tx=0x7f2050007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.690 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f2065ffb700 1 -- 172.21.15.38:0/2043688683 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2050011cd0 con 0x7f2068125840 2020-03-19T23:11:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 --2- 172.21.15.38:0/2043688683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2068125840 0x7f2068127c30 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7f20500032b0 tx=0x7f2050007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f20667fc700 1 --2- 172.21.15.38:0/2043688683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2068125840 0x7f2068127c30 secure :-1 s=THROTTLE_DONE pgs=67 cs=0 l=1 rx=0x7f20500032b0 tx=0x7f2050007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f2065ffb700 1 -- 172.21.15.38:0/2043688683 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2050011e30 con 0x7f2068125840 2020-03-19T23:11:01.691 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f2065ffb700 1 -- 172.21.15.38:0/2043688683 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f205000a290 con 0x7f2068125840 2020-03-19T23:11:01.692 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f206d78a700 1 -- 172.21.15.38:0/2043688683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2068125840 msgr2=0x7f2068127c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.692 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.672+0000 7f206d78a700 1 --2- 172.21.15.38:0/2043688683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2068125840 0x7f2068127c30 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f20500032b0 tx=0x7f2050007dc0).stop 2020-03-19T23:11:01.692 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.673+0000 7f206d78a700 1 -- 172.21.15.38:0/2043688683 shutdown_connections 2020-03-19T23:11:01.692 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.673+0000 7f206d78a700 1 --2- 172.21.15.38:0/2043688683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2068125840 0x7f2068127c30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.692 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.673+0000 7f206d78a700 1 -- 172.21.15.38:0/2043688683 >> 172.21.15.38:0/2043688683 conn(0x7f2068091390 msgr2=0x7f2068091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:01.694 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.673+0000 7f206d78a700 1 -- 172.21.15.38:0/2043688683 shutdown_connections 2020-03-19T23:11:01.694 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.673+0000 7f206d78a700 1 -- 172.21.15.38:0/2043688683 wait complete. 2020-03-19T23:11:01.694 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.673+0000 7f206d78a700 1 Processor -- start 2020-03-19T23:11:01.694 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.673+0000 7f206d78a700 1 -- start start 2020-03-19T23:11:01.695 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f206d78a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f20681224c0 0x7f206810aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:01.695 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f206d78a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206810dfd0 con 0x7f2068125840 2020-03-19T23:11:01.695 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f206d78a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206810ceb0 con 0x7f20681224c0 2020-03-19T23:11:01.695 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f20681224c0 0x7f206810aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.696 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f20681224c0 0x7f206810aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60660/0 (socket says 172.21.15.38:60660) 2020-03-19T23:11:01.696 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 -- 172.21.15.38:0/3102581444 learned_addr learned my addr 172.21.15.38:0/3102581444 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:01.696 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 -- 172.21.15.38:0/3102581444 >> v1:172.21.15.38:6789/0 conn(0x7f2068125840 legacy=0x7f206810b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.696 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473322284 0 0) 0x7f206810dfd0 con 0x7f2068125840 2020-03-19T23:11:01.697 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 -- 172.21.15.38:0/3102581444 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206810db40 con 0x7f20681224c0 2020-03-19T23:11:01.697 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> v2:172.21.15.38:3300/0 conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.697 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> v2:172.21.15.38:3300/0 conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.697 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.697 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.698 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2060005b20 con 0x7f20681224c0 2020-03-19T23:11:01.698 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.698 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.698 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060005c80 con 0x7f20681224c0 2020-03-19T23:11:01.699 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206810d830 con 0x7f20681224c0 2020-03-19T23:11:01.699 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f20600060c0 con 0x7f20681224c0 2020-03-19T23:11:01.699 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.674+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2068066220 con 0x7f20681224c0 2020-03-19T23:11:01.699 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.675+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.700 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.675+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f2060006330 con 0x7f20681224c0 2020-03-19T23:11:01.700 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.675+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.700 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.675+0000 7f205ffff700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 0x7f204c02a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:01.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.675+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7f2060033ee0 con 0x7f20681224c0 2020-03-19T23:11:01.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.675+0000 7f20667fc700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 0x7f204c02a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.676+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206810d000 con 0x7f20681224c0 2020-03-19T23:11:01.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.676+0000 7f20667fc700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 0x7f204c02a770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2050003250 tx=0x7f2050007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.676+0000 7f20667fc700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 0x7f204c02a770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2050003250 tx=0x7f2050007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.676+0000 7f20667fc700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 0x7f204c02a770 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f2050003250 tx=0x7f2050007dc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.679+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.703 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.679+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f20600090b0 con 0x7f20681224c0 2020-03-19T23:11:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.827+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2068056db0 con 0x7f20681224c0 2020-03-19T23:11:01.828 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.827+0000 7f2066ffd700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.829 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.827+0000 7f205ffff700 1 -- 172.21.15.38:0/3102581444 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v11) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2060010fc0 con 0x7f20681224c0 2020-03-19T23:11:01.829 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.828+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 msgr2=0x7f204c02a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.828+0000 7f206d78a700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 0x7f204c02a770 secure :-1 s=READY pgs=18 cs=0 l=1 rx=0x7f2050003250 tx=0x7f2050007dc0).stop 2020-03-19T23:11:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.828+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 msgr2=0x7f206810aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.828+0000 7f206d78a700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7f2060000b80 tx=0x7f2060010990).stop 2020-03-19T23:11:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.829+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 shutdown_connections 2020-03-19T23:11:01.830 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.829+0000 7f206d78a700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20681224c0 0x7f206810aed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.831 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.829+0000 7f206d78a700 1 --2- 172.21.15.38:0/3102581444 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f204c0282b0 0x7f204c02a770 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.831 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.829+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 >> 172.21.15.38:0/3102581444 conn(0x7f2068091390 msgr2=0x7f2068060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:01.831 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.829+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 shutdown_connections 2020-03-19T23:11:01.831 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.829+0000 7f206d78a700 1 -- 172.21.15.38:0/3102581444 wait complete. 2020-03-19T23:11:01.840 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early --format=json-pretty osd lspools 2020-03-19T23:11:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe21621d700 1 Processor -- start 2020-03-19T23:11:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe21621d700 1 -- start start 2020-03-19T23:11:01.962 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe21621d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe210090ed0 0x7fe21008e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe21621d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe210060a40 con 0x7fe2100903e0 2020-03-19T23:11:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe21621d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe210060500 con 0x7fe210090ed0 2020-03-19T23:11:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe214a1a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe210090ed0 0x7fe21008e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe21521b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe2100903e0 0x7fe2100907c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43806/0 (socket says 172.21.15.38:43806) 2020-03-19T23:11:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.952+0000 7fe21521b700 1 -- 172.21.15.38:0/215821023 learned_addr learned my addr 172.21.15.38:0/215821023 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 -- 172.21.15.38:0/215821023 >> v1:172.21.15.38:6789/0 conn(0x7fe2100903e0 legacy=0x7fe2100907c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 -- 172.21.15.38:0/215821023 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210060730 con 0x7fe210090ed0 2020-03-19T23:11:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 --2- 172.21.15.38:0/215821023 >> v2:172.21.15.38:3300/0 conn(0x7fe210090ed0 0x7fe21008e720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe20c002f30 tx=0x7fe20c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 --2- 172.21.15.38:0/215821023 >> v2:172.21.15.38:3300/0 conn(0x7fe210090ed0 0x7fe21008e720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe20c002f30 tx=0x7fe20c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 --2- 172.21.15.38:0/215821023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe210090ed0 0x7fe21008e720 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7fe20c002f30 tx=0x7fe20c007a40).ready entity=mon.0 client_cookie=7952a0a8da1ab08b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 --2- 172.21.15.38:0/215821023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe210090ed0 0x7fe21008e720 secure :-1 s=THROTTLE_DONE pgs=71 cs=0 l=1 rx=0x7fe20c002f30 tx=0x7fe20c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe203fff700 1 -- 172.21.15.38:0/215821023 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe20c008e80 con 0x7fe210090ed0 2020-03-19T23:11:01.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 --2- 172.21.15.38:0/215821023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe210090ed0 0x7fe21008e720 secure :-1 s=THROTTLE_DONE pgs=71 cs=0 l=1 rx=0x7fe20c002f30 tx=0x7fe20c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe203fff700 1 -- 172.21.15.38:0/215821023 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c0113f0 con 0x7fe210090ed0 2020-03-19T23:11:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe214a1a700 1 --2- 172.21.15.38:0/215821023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe210090ed0 0x7fe21008e720 secure :-1 s=THROTTLE_DONE pgs=71 cs=0 l=1 rx=0x7fe20c002f30 tx=0x7fe20c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe21621d700 1 -- 172.21.15.38:0/215821023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe210090ed0 msgr2=0x7fe21008e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe203fff700 1 -- 172.21.15.38:0/215821023 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe20c011710 con 0x7fe210090ed0 2020-03-19T23:11:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe21621d700 1 --2- 172.21.15.38:0/215821023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe210090ed0 0x7fe21008e720 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7fe20c002f30 tx=0x7fe20c007a40).stop 2020-03-19T23:11:01.965 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe21621d700 1 -- 172.21.15.38:0/215821023 shutdown_connections 2020-03-19T23:11:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe21621d700 1 --2- 172.21.15.38:0/215821023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe210090ed0 0x7fe21008e720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.953+0000 7fe21621d700 1 -- 172.21.15.38:0/215821023 >> 172.21.15.38:0/215821023 conn(0x7fe2100953a0 msgr2=0x7fe2100914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.954+0000 7fe21621d700 1 -- 172.21.15.38:0/215821023 shutdown_connections 2020-03-19T23:11:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.954+0000 7fe21621d700 1 -- 172.21.15.38:0/215821023 wait complete. 2020-03-19T23:11:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.954+0000 7fe21621d700 1 Processor -- start 2020-03-19T23:11:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.954+0000 7fe21621d700 1 -- start start 2020-03-19T23:11:01.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21621d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe2100903e0 0x7fe210131140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21621d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100668c0 con 0x7fe210090ed0 2020-03-19T23:11:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21621d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe210066b80 con 0x7fe2100903e0 2020-03-19T23:11:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe2100903e0 0x7fe210131140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe214a1a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe210090ed0 0x7fe2101315a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43812/0 (socket says 172.21.15.38:43812) 2020-03-19T23:11:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe214a1a700 1 -- 172.21.15.38:0/186282920 learned_addr learned my addr 172.21.15.38:0/186282920 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 -- 172.21.15.38:0/186282920 >> v1:172.21.15.38:6789/0 conn(0x7fe210090ed0 legacy=0x7fe2101315a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:01.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 -- 172.21.15.38:0/186282920 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210064940 con 0x7fe2100903e0 2020-03-19T23:11:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981774882 0 0) 0x7fe2100668c0 con 0x7fe210090ed0 2020-03-19T23:11:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> v2:172.21.15.38:3300/0 conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> v2:172.21.15.38:3300/0 conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe204005b20 con 0x7fe2100903e0 2020-03-19T23:11:01.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204005c80 con 0x7fe2100903e0 2020-03-19T23:11:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe2040060c0 con 0x7fe2100903e0 2020-03-19T23:11:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.955+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe210065620 con 0x7fe2100903e0 2020-03-19T23:11:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2100651e0 con 0x7fe2100903e0 2020-03-19T23:11:01.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fe20400a320 con 0x7fe2100903e0 2020-03-19T23:11:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe2027fc700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 0x7fe1f002a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7fe204033fc0 con 0x7fe2100903e0 2020-03-19T23:11:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe214a1a700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 0x7fe1f002a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe214a1a700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 0x7fe1f002a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe20c0101c0 tx=0x7fe20c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe210056db0 con 0x7fe2100903e0 2020-03-19T23:11:01.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe214a1a700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 0x7fe1f002a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe20c0101c0 tx=0x7fe20c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.956+0000 7fe214a1a700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 0x7fe1f002a710 secure :-1 s=READY pgs=19 cs=0 l=1 rx=0x7fe20c0101c0 tx=0x7fe20c007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:01.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.960+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:01.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:01.960+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fe204006220 con 0x7fe2100903e0 2020-03-19T23:11:02.114 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.110+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe21006aaf0 con 0x7fe2100903e0 2020-03-19T23:11:02.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.111+0000 7fe21521b700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.111+0000 7fe2027fc700 1 -- 172.21.15.38:0/186282920 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v11) v1 ==== 84+0+64 (secure 0 0 0) 0x7fe204011d60 con 0x7fe2100903e0 2020-03-19T23:11:02.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 msgr2=0x7fe1f002a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:02.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 0x7fe1f002a710 secure :-1 s=READY pgs=19 cs=0 l=1 rx=0x7fe20c0101c0 tx=0x7fe20c007a40).stop 2020-03-19T23:11:02.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 msgr2=0x7fe210131140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:02.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7fe2100446e0 tx=0x7fe2040108c0).stop 2020-03-19T23:11:02.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 shutdown_connections 2020-03-19T23:11:02.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2100903e0 0x7fe210131140 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:02.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 --2- 172.21.15.38:0/186282920 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe1f0028250 0x7fe1f002a710 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:02.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 >> 172.21.15.38:0/186282920 conn(0x7fe2100953a0 msgr2=0x7fe21008bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:02.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 shutdown_connections 2020-03-19T23:11:02.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.112+0000 7fe21621d700 1 -- 172.21.15.38:0/186282920 wait complete. 2020-03-19T23:11:02.117 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:02.122 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2020-03-19T23:11:02.122 INFO:teuthology.orchestra.run.smithi038.stdout: { 2020-03-19T23:11:02.122 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2020-03-19T23:11:02.122 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": "rbd" 2020-03-19T23:11:02.123 INFO:teuthology.orchestra.run.smithi038.stdout: } 2020-03-19T23:11:02.123 INFO:teuthology.orchestra.run.smithi038.stdout:] 2020-03-19T23:11:02.124 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-03-19T23:11:02.124 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_metadata 8 2020-03-19T23:11:02.241 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.240+0000 7fbc48405700 1 Processor -- start 2020-03-19T23:11:02.241 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.240+0000 7fbc48405700 1 -- start start 2020-03-19T23:11:02.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc48405700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc40125060 0x7fbc40127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc48405700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc40060f10 con 0x7fbc40065970 2020-03-19T23:11:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc48405700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc40061490 con 0x7fbc40125060 2020-03-19T23:11:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc40125060 0x7fbc40127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc461a1700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbc40065970 0x7fbc40065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43816/0 (socket says 172.21.15.38:43816) 2020-03-19T23:11:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc461a1700 1 -- 172.21.15.38:0/3737043589 learned_addr learned my addr 172.21.15.38:0/3737043589 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:02.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc40125060 0x7fbc40127450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60678/0 (socket says 172.21.15.38:60678) 2020-03-19T23:11:02.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 -- 172.21.15.38:0/3737043589 >> v1:172.21.15.38:6789/0 conn(0x7fbc40065970 legacy=0x7fbc40065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:02.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 -- 172.21.15.38:0/3737043589 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc40061730 con 0x7fbc40125060 2020-03-19T23:11:02.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- 172.21.15.38:0/3737043589 >> v2:172.21.15.38:3300/0 conn(0x7fbc40125060 0x7fbc40127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fbc3c002f30 tx=0x7fbc3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- 172.21.15.38:0/3737043589 >> v2:172.21.15.38:3300/0 conn(0x7fbc40125060 0x7fbc40127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fbc3c002f30 tx=0x7fbc3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- 172.21.15.38:0/3737043589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40125060 0x7fbc40127450 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7fbc3c002f30 tx=0x7fbc3c007a40).ready entity=mon.0 client_cookie=e2cefe8bfc1af341 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:02.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- 172.21.15.38:0/3737043589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40125060 0x7fbc40127450 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7fbc3c002f30 tx=0x7fbc3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc4519f700 1 -- 172.21.15.38:0/3737043589 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbc3c0113f0 con 0x7fbc40125060 2020-03-19T23:11:02.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- 172.21.15.38:0/3737043589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40125060 0x7fbc40127450 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7fbc3c002f30 tx=0x7fbc3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc459a0700 1 --2- 172.21.15.38:0/3737043589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40125060 0x7fbc40127450 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7fbc3c002f30 tx=0x7fbc3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc4519f700 1 -- 172.21.15.38:0/3737043589 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c011550 con 0x7fbc40125060 2020-03-19T23:11:02.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc4519f700 1 -- 172.21.15.38:0/3737043589 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbc3c011870 con 0x7fbc40125060 2020-03-19T23:11:02.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc48405700 1 -- 172.21.15.38:0/3737043589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40125060 msgr2=0x7fbc40127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:02.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.241+0000 7fbc48405700 1 --2- 172.21.15.38:0/3737043589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40125060 0x7fbc40127450 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7fbc3c002f30 tx=0x7fbc3c007a40).stop 2020-03-19T23:11:02.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.242+0000 7fbc48405700 1 -- 172.21.15.38:0/3737043589 shutdown_connections 2020-03-19T23:11:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.242+0000 7fbc48405700 1 --2- 172.21.15.38:0/3737043589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40125060 0x7fbc40127450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.242+0000 7fbc48405700 1 -- 172.21.15.38:0/3737043589 >> 172.21.15.38:0/3737043589 conn(0x7fbc40091390 msgr2=0x7fbc40091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.242+0000 7fbc48405700 1 -- 172.21.15.38:0/3737043589 shutdown_connections 2020-03-19T23:11:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.242+0000 7fbc48405700 1 -- 172.21.15.38:0/3737043589 wait complete. 2020-03-19T23:11:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc48405700 1 Processor -- start 2020-03-19T23:11:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc48405700 1 -- start start 2020-03-19T23:11:02.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc48405700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc40065970 0x7fbc4012ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc48405700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc40111390 con 0x7fbc40125060 2020-03-19T23:11:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc48405700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc401257f0 con 0x7fbc40065970 2020-03-19T23:11:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc461a1700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc40065970 0x7fbc4012ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc459a0700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbc40125060 0x7fbc4010de90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43822/0 (socket says 172.21.15.38:43822) 2020-03-19T23:11:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc459a0700 1 -- 172.21.15.38:0/120182783 learned_addr learned my addr 172.21.15.38:0/120182783 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc461a1700 1 -- 172.21.15.38:0/120182783 >> v1:172.21.15.38:6789/0 conn(0x7fbc40125060 legacy=0x7fbc4010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:02.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc461a1700 1 -- 172.21.15.38:0/120182783 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc401254e0 con 0x7fbc40065970 2020-03-19T23:11:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963793497 0 0) 0x7fbc40111390 con 0x7fbc40125060 2020-03-19T23:11:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.243+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> v2:172.21.15.38:3300/0 conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> v2:172.21.15.38:3300/0 conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbc30005b20 con 0x7fbc40065970 2020-03-19T23:11:02.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc30005c80 con 0x7fbc40065970 2020-03-19T23:11:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4010f040 con 0x7fbc40065970 2020-03-19T23:11:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbc300060c0 con 0x7fbc40065970 2020-03-19T23:11:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc40110690 con 0x7fbc40065970 2020-03-19T23:11:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fbc3000a320 con 0x7fbc40065970 2020-03-19T23:11:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.244+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.245+0000 7fbc2f7fe700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 0x7fbc2002a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.245+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2583+0+0 (secure 0 0 0) 0x7fbc30033e60 con 0x7fbc40065970 2020-03-19T23:11:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.245+0000 7fbc459a0700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 0x7fbc2002a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.245+0000 7fbc459a0700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 0x7fbc2002a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbc3c0101c0 tx=0x7fbc3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.245+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc40060f10 con 0x7fbc40065970 2020-03-19T23:11:02.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.245+0000 7fbc459a0700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 0x7fbc2002a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbc3c0101c0 tx=0x7fbc3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.256 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.245+0000 7fbc459a0700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 0x7fbc2002a740 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7fbc3c0101c0 tx=0x7fbc3c007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.249+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:02.257 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.249+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fbc40060f10 con 0x7fbc40065970 2020-03-19T23:11:02.397 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:02.396+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fbc40060f10 con 0x7fbc40065970 2020-03-19T23:11:03.244 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.243+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.276 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.275+0000 7fbc461a1700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=THROTTLE_DONE pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.276 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.275+0000 7fbc2f7fe700 1 -- 172.21.15.38:0/120182783 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v13) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbc40060f10 con 0x7fbc40065970 2020-03-19T23:11:03.279 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 msgr2=0x7fbc2002a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:03.279 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 0x7fbc2002a740 secure :-1 s=READY pgs=20 cs=0 l=1 rx=0x7fbc3c0101c0 tx=0x7fbc3c007a40).stop 2020-03-19T23:11:03.279 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 msgr2=0x7fbc4012ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:03.280 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7fbc4008ee80 tx=0x7fbc300108a0).stop 2020-03-19T23:11:03.280 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 shutdown_connections 2020-03-19T23:11:03.280 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc40065970 0x7fbc4012ea80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:03.280 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 --2- 172.21.15.38:0/120182783 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbc20028280 0x7fbc2002a740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:03.280 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 >> 172.21.15.38:0/120182783 conn(0x7fbc40091390 msgr2=0x7fbc40093250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:03.280 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 shutdown_connections 2020-03-19T23:11:03.280 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.278+0000 7fbc48405700 1 -- 172.21.15.38:0/120182783 wait complete. 2020-03-19T23:11:03.281 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2020-03-19T23:11:03.293 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create cephfs_data 8 2020-03-19T23:11:03.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.415+0000 7f72bffff700 1 Processor -- start 2020-03-19T23:11:03.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.415+0000 7f72bffff700 1 -- start start 2020-03-19T23:11:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72c0066bf0 0x7f72c01206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bffff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c0062660 con 0x7f72c0120b20 2020-03-19T23:11:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bffff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c0090980 con 0x7f72c0066bf0 2020-03-19T23:11:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72c0066bf0 0x7f72c01206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72be7fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f72c0120b20 0x7f72c008f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43828/0 (socket says 172.21.15.38:43828) 2020-03-19T23:11:03.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72be7fc700 1 -- 172.21.15.38:0/2936401194 learned_addr learned my addr 172.21.15.38:0/2936401194 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 -- 172.21.15.38:0/2936401194 >> v1:172.21.15.38:6789/0 conn(0x7f72c0120b20 legacy=0x7f72c008f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 -- 172.21.15.38:0/2936401194 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0066fd0 con 0x7f72c0066bf0 2020-03-19T23:11:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bdffb700 1 -- 172.21.15.38:0/2936401194 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378702303 0 0) 0x7f72b00010d0 con 0x7f72c0120b20 2020-03-19T23:11:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 --2- 172.21.15.38:0/2936401194 >> v2:172.21.15.38:3300/0 conn(0x7f72c0066bf0 0x7f72c01206c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f72b4002f30 tx=0x7f72b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 --2- 172.21.15.38:0/2936401194 >> v2:172.21.15.38:3300/0 conn(0x7f72c0066bf0 0x7f72c01206c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f72b4002f30 tx=0x7f72b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 --2- 172.21.15.38:0/2936401194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0066bf0 0x7f72c01206c0 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7f72b4002f30 tx=0x7f72b4007a40).ready entity=mon.0 client_cookie=9258ad282e3fe738 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:03.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 --2- 172.21.15.38:0/2936401194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0066bf0 0x7f72c01206c0 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7f72b4002f30 tx=0x7f72b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bdffb700 1 -- 172.21.15.38:0/2936401194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72b4008e80 con 0x7f72c0066bf0 2020-03-19T23:11:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 --2- 172.21.15.38:0/2936401194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0066bf0 0x7f72c01206c0 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7f72b4002f30 tx=0x7f72b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72beffd700 1 --2- 172.21.15.38:0/2936401194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0066bf0 0x7f72c01206c0 secure :-1 s=THROTTLE_DONE pgs=79 cs=0 l=1 rx=0x7f72b4002f30 tx=0x7f72b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bdffb700 1 -- 172.21.15.38:0/2936401194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b40113f0 con 0x7f72c0066bf0 2020-03-19T23:11:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bdffb700 1 -- 172.21.15.38:0/2936401194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72b40116d0 con 0x7f72c0066bf0 2020-03-19T23:11:03.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bffff700 1 -- 172.21.15.38:0/2936401194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0066bf0 msgr2=0x7f72c01206c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:03.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.416+0000 7f72bffff700 1 --2- 172.21.15.38:0/2936401194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0066bf0 0x7f72c01206c0 secure :-1 s=READY pgs=79 cs=0 l=1 rx=0x7f72b4002f30 tx=0x7f72b4007a40).stop 2020-03-19T23:11:03.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.417+0000 7f72bffff700 1 -- 172.21.15.38:0/2936401194 shutdown_connections 2020-03-19T23:11:03.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.417+0000 7f72bffff700 1 --2- 172.21.15.38:0/2936401194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0066bf0 0x7f72c01206c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:03.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.417+0000 7f72bffff700 1 -- 172.21.15.38:0/2936401194 >> 172.21.15.38:0/2936401194 conn(0x7f72c00950e0 msgr2=0x7f72c0091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:03.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.417+0000 7f72bffff700 1 -- 172.21.15.38:0/2936401194 shutdown_connections 2020-03-19T23:11:03.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.417+0000 7f72bffff700 1 -- 172.21.15.38:0/2936401194 wait complete. 2020-03-19T23:11:03.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72bffff700 1 Processor -- start 2020-03-19T23:11:03.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72bffff700 1 -- start start 2020-03-19T23:11:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72bffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72c0120b20 0x7f72c011cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72bffff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c010ccd0 con 0x7f72c0066bf0 2020-03-19T23:11:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72bffff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c0065080 con 0x7f72c0120b20 2020-03-19T23:11:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72beffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f72c0066bf0 0x7f72c011c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43830/0 (socket says 172.21.15.38:43830) 2020-03-19T23:11:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72beffd700 1 -- 172.21.15.38:0/3667420522 learned_addr learned my addr 172.21.15.38:0/3667420522 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.418+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> v2:172.21.15.38:3300/0 conn(0x7f72c0120b20 0x7f72c011cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635030967 0 0) 0x7f72c010ccd0 con 0x7f72c0066bf0 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f729c002fe0 con 0x7f72c0066bf0 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 -- 172.21.15.38:0/3667420522 >> v1:172.21.15.38:6789/0 conn(0x7f72c0066bf0 legacy=0x7f72c011c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 -- 172.21.15.38:0/3667420522 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c010d580 con 0x7f72c0120b20 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> v2:172.21.15.38:3300/0 conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> v2:172.21.15.38:3300/0 conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=READY pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72b0005b20 con 0x7f72c0120b20 2020-03-19T23:11:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0005c80 con 0x7f72c0120b20 2020-03-19T23:11:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c010d270 con 0x7f72c0120b20 2020-03-19T23:11:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72b00060c0 con 0x7f72c0120b20 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c00653e0 con 0x7f72c0120b20 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.419+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f72b0006340 con 0x7f72c0120b20 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72affff700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 0x7f729c02ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2937+0+0 (secure 0 0 0) 0x7f72b0033ed0 con 0x7f72c0120b20 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72beffd700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 0x7f729c02ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:03.430 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c0062660 con 0x7f72c0120b20 2020-03-19T23:11:03.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72beffd700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 0x7f729c02ac20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f72b40088e0 tx=0x7f72b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72beffd700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 0x7f729c02ac20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f72b40088e0 tx=0x7f72b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.420+0000 7f72beffd700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 0x7f729c02ac20 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7f72b40088e0 tx=0x7f72b4007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:03.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.424+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:03.431 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.424+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f72c0062660 con 0x7f72c0120b20 2020-03-19T23:11:03.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:03.570+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f72c0062660 con 0x7f72c0120b20 2020-03-19T23:11:04.278 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.277+0000 7f72be7fc700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=THROTTLE_DONE pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.278 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.277+0000 7f72affff700 1 -- 172.21.15.38:0/3667420522 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v14) v1 ==== 125+0+0 (secure 0 0 0) 0x7f72c0062660 con 0x7f72c0120b20 2020-03-19T23:11:04.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.285+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 msgr2=0x7f729c02ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.285+0000 7f72bffff700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 0x7f729c02ac20 secure :-1 s=READY pgs=21 cs=0 l=1 rx=0x7f72b40088e0 tx=0x7f72b4007a40).stop 2020-03-19T23:11:04.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.285+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 msgr2=0x7f72c011cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.285+0000 7f72bffff700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 secure :-1 s=READY pgs=81 cs=0 l=1 rx=0x7f72b0000b80 tx=0x7f72b00108a0).stop 2020-03-19T23:11:04.288 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.287+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 shutdown_connections 2020-03-19T23:11:04.288 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.287+0000 7f72bffff700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f729c028760 0x7f729c02ac20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.288 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.287+0000 7f72bffff700 1 --2- 172.21.15.38:0/3667420522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72c0120b20 0x7f72c011cf80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.288 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.287+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 >> 172.21.15.38:0/3667420522 conn(0x7f72c00950e0 msgr2=0x7f72c011f270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:04.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.288+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 shutdown_connections 2020-03-19T23:11:04.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.288+0000 7f72bffff700 1 -- 172.21.15.38:0/3667420522 wait complete. 2020-03-19T23:11:04.290 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2020-03-19T23:11:04.302 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs new cephfs cephfs_metadata cephfs_data --force 2020-03-19T23:11:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.425+0000 7f33faec4700 1 Processor -- start 2020-03-19T23:11:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.425+0000 7f33faec4700 1 -- start start 2020-03-19T23:11:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33faec4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f408f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:04.438 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33faec4700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f4090c70 con 0x7f33f4122950 2020-03-19T23:11:04.439 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33faec4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f4090fc0 con 0x7f33f4122d30 2020-03-19T23:11:04.439 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f408f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:04.439 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f8c60700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f33f4122950 0x7f33f408f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43836/0 (socket says 172.21.15.38:43836) 2020-03-19T23:11:04.439 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f408f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60698/0 (socket says 172.21.15.38:60698) 2020-03-19T23:11:04.440 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f8c60700 1 -- 172.21.15.38:0/2543688633 learned_addr learned my addr 172.21.15.38:0/2543688633 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:04.440 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 -- 172.21.15.38:0/2543688633 >> v1:172.21.15.38:6789/0 conn(0x7f33f4122950 legacy=0x7f33f408f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.440 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 -- 172.21.15.38:0/2543688633 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4122750 con 0x7f33f4122d30 2020-03-19T23:11:04.440 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f37fe700 1 -- 172.21.15.38:0/2543688633 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092708700 0 0) 0x7f33f4090c70 con 0x7f33f4122950 2020-03-19T23:11:04.441 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- 172.21.15.38:0/2543688633 >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f408f9a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f33f4066d30 tx=0x7f33e00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.441 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- 172.21.15.38:0/2543688633 >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f408f9a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f33f4066d30 tx=0x7f33e00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.441 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- 172.21.15.38:0/2543688633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f408f9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33e00088e0).ready entity=mon.0 client_cookie=11f3a430dd72dd3b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:04.441 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- 172.21.15.38:0/2543688633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f408f9a0 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33e00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f37fe700 1 -- 172.21.15.38:0/2543688633 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f33e00113f0 con 0x7f33f4122d30 2020-03-19T23:11:04.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- 172.21.15.38:0/2543688633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f408f9a0 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33e00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f37fe700 1 -- 172.21.15.38:0/2543688633 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0011550 con 0x7f33f4122d30 2020-03-19T23:11:04.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f3fff700 1 --2- 172.21.15.38:0/2543688633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f408f9a0 secure :-1 s=THROTTLE_DONE pgs=83 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33e00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33faec4700 1 -- 172.21.15.38:0/2543688633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 msgr2=0x7f33f408f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33faec4700 1 --2- 172.21.15.38:0/2543688633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f408f9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33e00088e0).stop 2020-03-19T23:11:04.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.426+0000 7f33f37fe700 1 -- 172.21.15.38:0/2543688633 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f33e0011830 con 0x7f33f4122d30 2020-03-19T23:11:04.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.427+0000 7f33faec4700 1 -- 172.21.15.38:0/2543688633 shutdown_connections 2020-03-19T23:11:04.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.427+0000 7f33faec4700 1 --2- 172.21.15.38:0/2543688633 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f408f9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.427+0000 7f33faec4700 1 -- 172.21.15.38:0/2543688633 >> 172.21.15.38:0/2543688633 conn(0x7f33f4091390 msgr2=0x7f33f4091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:04.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.427+0000 7f33faec4700 1 -- 172.21.15.38:0/2543688633 shutdown_connections 2020-03-19T23:11:04.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.427+0000 7f33faec4700 1 -- 172.21.15.38:0/2543688633 wait complete. 2020-03-19T23:11:04.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.427+0000 7f33faec4700 1 Processor -- start 2020-03-19T23:11:04.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.427+0000 7f33faec4700 1 -- start start 2020-03-19T23:11:04.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33faec4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f41332d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:04.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33faec4700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f41328f0 con 0x7f33f4122950 2020-03-19T23:11:04.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33faec4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f4110350 con 0x7f33f4122d30 2020-03-19T23:11:04.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f41332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:04.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f8c60700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f33f4122950 0x7f33f4132bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43840/0 (socket says 172.21.15.38:43840) 2020-03-19T23:11:04.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f41332d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60702/0 (socket says 172.21.15.38:60702) 2020-03-19T23:11:04.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f8c60700 1 -- 172.21.15.38:0/3810951784 learned_addr learned my addr 172.21.15.38:0/3810951784 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:04.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f3fff700 1 -- 172.21.15.38:0/3810951784 >> v1:172.21.15.38:6789/0 conn(0x7f33f4122950 legacy=0x7f33f4132bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f3fff700 1 -- 172.21.15.38:0/3810951784 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f41124a0 con 0x7f33f4122d30 2020-03-19T23:11:04.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517101371 0 0) 0x7f33f41328f0 con 0x7f33f4122950 2020-03-19T23:11:04.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.428+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> v2:172.21.15.38:3300/0 conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=READY pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:04.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f33e000f740 con 0x7f33f4122d30 2020-03-19T23:11:04.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e00056c0 con 0x7f33f4122d30 2020-03-19T23:11:04.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f4110a60 con 0x7f33f4122d30 2020-03-19T23:11:04.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f33e00059a0 con 0x7f33f4122d30 2020-03-19T23:11:04.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f410e750 con 0x7f33f4122d30 2020-03-19T23:11:04.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f33e0005ba0 con 0x7f33f4122d30 2020-03-19T23:11:04.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.429+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.430+0000 7f33f1ffb700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 0x7f33dc02a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:04.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.430+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3287+0+0 (secure 0 0 0) 0x7f33e0034770 con 0x7f33f4122d30 2020-03-19T23:11:04.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.430+0000 7f33f8c60700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 0x7f33dc02a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:04.452 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.430+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f41101c0 con 0x7f33f4122d30 2020-03-19T23:11:04.452 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.430+0000 7f33f8c60700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 0x7f33dc02a720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f33e8005560 tx=0x7f33e80016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.452 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.430+0000 7f33f8c60700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 0x7f33dc02a720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f33e8005560 tx=0x7f33e80016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.452 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.430+0000 7f33f8c60700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 0x7f33dc02a720 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f33e8005560 tx=0x7f33e80016c0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:04.453 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.434+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.453 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.434+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f33e0005e30 con 0x7f33f4122d30 2020-03-19T23:11:04.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.633+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f33f4056db0 con 0x7f33f4122d30 2020-03-19T23:11:04.635 INFO:tasks.ceph.mon.a.smithi038.stderr:2020-03-19T23:11:04.634+0000 7f82d661e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-03-19T23:11:04.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.646+0000 7f33f3fff700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=THROTTLE_DONE pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.646+0000 7f33f1ffb700 1 -- 172.21.15.38:0/3810951784 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 2 and data pool 3 v2) v1 ==== 187+0+0 (secure 0 0 0) 0x7f33e00090c0 con 0x7f33f4122d30 2020-03-19T23:11:04.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.650+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 msgr2=0x7f33dc02a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.650+0000 7f33faec4700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 0x7f33dc02a720 secure :-1 s=READY pgs=22 cs=0 l=1 rx=0x7f33e8005560 tx=0x7f33e80016c0).stop 2020-03-19T23:11:04.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.650+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 msgr2=0x7f33f41332d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.650+0000 7f33faec4700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 secure :-1 s=READY pgs=85 cs=0 l=1 rx=0x7f33f4066d30 tx=0x7f33f4094810).stop 2020-03-19T23:11:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.651+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 shutdown_connections 2020-03-19T23:11:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.651+0000 7f33faec4700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f33dc028260 0x7f33dc02a720 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.651+0000 7f33faec4700 1 --2- 172.21.15.38:0/3810951784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f4122d30 0x7f33f41332d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.651+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 >> 172.21.15.38:0/3810951784 conn(0x7f33f4091390 msgr2=0x7f33f4061ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.651+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 shutdown_connections 2020-03-19T23:11:04.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.651+0000 7f33faec4700 1 -- 172.21.15.38:0/3810951784 wait complete. 2020-03-19T23:11:04.654 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2020-03-19T23:11:04.663 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:04.782 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.775+0000 7ff498bf7700 1 Processor -- start 2020-03-19T23:11:04.782 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.775+0000 7ff498bf7700 1 -- start start 2020-03-19T23:11:04.783 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff498bf7700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff494127f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:04.783 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff498bf7700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff494090af0 con 0x7ff494065d30 2020-03-19T23:11:04.783 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff498bf7700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff494066820 con 0x7ff494125ba0 2020-03-19T23:11:04.783 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff48bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff494127f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:04.783 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff49259c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff494065d30 0x7ff494066110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43846/0 (socket says 172.21.15.38:43846) 2020-03-19T23:11:04.783 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff49259c700 1 -- 172.21.15.38:0/1955957787 learned_addr learned my addr 172.21.15.38:0/1955957787 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:04.784 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff48bfff700 1 --2- 172.21.15.38:0/1955957787 >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff494127f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60708/0 (socket says 172.21.15.38:60708) 2020-03-19T23:11:04.784 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff48bfff700 1 -- 172.21.15.38:0/1955957787 >> v1:172.21.15.38:6789/0 conn(0x7ff494065d30 legacy=0x7ff494066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.784 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff491d9b700 1 -- 172.21.15.38:0/1955957787 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220492213 0 0) 0x7ff494090af0 con 0x7ff494065d30 2020-03-19T23:11:04.784 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff48bfff700 1 -- 172.21.15.38:0/1955957787 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49408eec0 con 0x7ff494125ba0 2020-03-19T23:11:04.785 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff48bfff700 1 --2- 172.21.15.38:0/1955957787 >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff494127f90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ff47c002f30 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.785 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff48bfff700 1 --2- 172.21.15.38:0/1955957787 >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff494127f90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ff47c002f30 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.785 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.776+0000 7ff48bfff700 1 --2- 172.21.15.38:0/1955957787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff494127f90 secure :-1 s=READY pgs=87 cs=0 l=1 rx=0x7ff47c002f30 tx=0x7ff47c007a40).ready entity=mon.0 client_cookie=9eff8152b50c0ea server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:04.785 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff48bfff700 1 --2- 172.21.15.38:0/1955957787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff494127f90 secure :-1 s=THROTTLE_DONE pgs=87 cs=0 l=1 rx=0x7ff47c002f30 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff491d9b700 1 -- 172.21.15.38:0/1955957787 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff47c008e80 con 0x7ff494125ba0 2020-03-19T23:11:04.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff48bfff700 1 --2- 172.21.15.38:0/1955957787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff494127f90 secure :-1 s=THROTTLE_DONE pgs=87 cs=0 l=1 rx=0x7ff47c002f30 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff48bfff700 1 --2- 172.21.15.38:0/1955957787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff494127f90 secure :-1 s=THROTTLE_DONE pgs=87 cs=0 l=1 rx=0x7ff47c002f30 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff491d9b700 1 -- 172.21.15.38:0/1955957787 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c0113f0 con 0x7ff494125ba0 2020-03-19T23:11:04.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff491d9b700 1 -- 172.21.15.38:0/1955957787 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff47c0116d0 con 0x7ff494125ba0 2020-03-19T23:11:04.787 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff498bf7700 1 -- 172.21.15.38:0/1955957787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 msgr2=0x7ff494127f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.787 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff498bf7700 1 --2- 172.21.15.38:0/1955957787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff494127f90 secure :-1 s=READY pgs=87 cs=0 l=1 rx=0x7ff47c002f30 tx=0x7ff47c007a40).stop 2020-03-19T23:11:04.787 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff498bf7700 1 -- 172.21.15.38:0/1955957787 shutdown_connections 2020-03-19T23:11:04.788 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff498bf7700 1 --2- 172.21.15.38:0/1955957787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff494127f90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.788 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff498bf7700 1 -- 172.21.15.38:0/1955957787 >> 172.21.15.38:0/1955957787 conn(0x7ff494091540 msgr2=0x7ff494091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:04.788 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff498bf7700 1 -- 172.21.15.38:0/1955957787 shutdown_connections 2020-03-19T23:11:04.788 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.777+0000 7ff498bf7700 1 -- 172.21.15.38:0/1955957787 wait complete. 2020-03-19T23:11:04.788 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff498bf7700 1 Processor -- start 2020-03-19T23:11:04.788 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff498bf7700 1 -- start start 2020-03-19T23:11:04.788 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff498bf7700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff49411df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:04.789 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff498bf7700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49408cc00 con 0x7ff494065d30 2020-03-19T23:11:04.789 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff498bf7700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49408c2e0 con 0x7ff494125ba0 2020-03-19T23:11:04.789 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff49259c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff494065d30 0x7ff49411d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43850/0 (socket says 172.21.15.38:43850) 2020-03-19T23:11:04.789 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff49259c700 1 -- 172.21.15.38:0/2818330144 learned_addr learned my addr 172.21.15.38:0/2818330144 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:04.789 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff49411df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:04.789 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.778+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390052808 0 0) 0x7ff49408cc00 con 0x7ff494065d30 2020-03-19T23:11:04.790 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff478002fe0 con 0x7ff494065d30 2020-03-19T23:11:04.790 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 -- 172.21.15.38:0/2818330144 >> v1:172.21.15.38:6789/0 conn(0x7ff494065d30 legacy=0x7ff49411d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.790 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 -- 172.21.15.38:0/2818330144 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4941256a0 con 0x7ff494125ba0 2020-03-19T23:11:04.790 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.790 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> v2:172.21.15.38:3300/0 conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.790 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=READY pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:04.791 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.791 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.791 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff47c00f830 con 0x7ff494125ba0 2020-03-19T23:11:04.791 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.791 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c007150 con 0x7ff494125ba0 2020-03-19T23:11:04.791 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49411f0d0 con 0x7ff494125ba0 2020-03-19T23:11:04.792 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff47c0073f0 con 0x7ff494125ba0 2020-03-19T23:11:04.792 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4941203b0 con 0x7ff494125ba0 2020-03-19T23:11:04.792 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.779+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.792 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7ff47c0056c0 con 0x7ff494125ba0 2020-03-19T23:11:04.792 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.793 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff48b7fe700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 0x7ff47802ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:04.793 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3403+0+0 (secure 0 0 0) 0x7ff47c034a50 con 0x7ff494125ba0 2020-03-19T23:11:04.793 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff49259c700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 0x7ff47802ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:04.793 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff49259c700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 0x7ff47802ac80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff484005520 tx=0x7ff4840016a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.793 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff49259c700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 0x7ff47802ac80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff484005520 tx=0x7ff4840016a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.793 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff49259c700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 0x7ff47802ac80 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7ff484005520 tx=0x7ff4840016a0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:04.793 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.780+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff494056db0 con 0x7ff494125ba0 2020-03-19T23:11:04.794 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.784+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.794 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.784+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7ff47c006250 con 0x7ff494125ba0 2020-03-19T23:11:04.935 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.933+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff49411faf0 con 0x7ff494125ba0 2020-03-19T23:11:04.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.934+0000 7ff48bfff700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=THROTTLE_DONE pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:04.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.934+0000 7ff48b7fe700 1 -- 172.21.15.38:0/2818330144 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8814 (secure 0 0 0) 0x7ff47c006070 con 0x7ff494125ba0 2020-03-19T23:11:04.941 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 msgr2=0x7ff47802ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.941 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 0x7ff47802ac80 secure :-1 s=READY pgs=23 cs=0 l=1 rx=0x7ff484005520 tx=0x7ff4840016a0).stop 2020-03-19T23:11:04.943 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 msgr2=0x7ff49411df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:04.943 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 secure :-1 s=READY pgs=89 cs=0 l=1 rx=0x7ff49411f7b0 tx=0x7ff47c007a40).stop 2020-03-19T23:11:04.943 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 shutdown_connections 2020-03-19T23:11:04.944 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff4780287c0 0x7ff47802ac80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.944 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 --2- 172.21.15.38:0/2818330144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff494125ba0 0x7ff49411df70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:04.944 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 >> 172.21.15.38:0/2818330144 conn(0x7ff494091540 msgr2=0x7ff494093500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:04.944 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 shutdown_connections 2020-03-19T23:11:04.945 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:04.936+0000 7ff498bf7700 1 -- 172.21.15.38:0/2818330144 wait complete. 2020-03-19T23:11:04.945 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:04.945 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":15,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:04.634342+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:04.947 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:05.067 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.059+0000 7f974c5de700 1 Processor -- start 2020-03-19T23:11:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.060+0000 7f974c5de700 1 -- start start 2020-03-19T23:11:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.060+0000 7f974c5de700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f974408d000 0x7f97440907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.060+0000 7f974c5de700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97440605e0 con 0x7f974408d3e0 2020-03-19T23:11:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.060+0000 7f974c5de700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974408ef70 con 0x7f974408d000 2020-03-19T23:11:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.060+0000 7f974a37a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f974408d000 0x7f97440907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:05.068 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.060+0000 7f9749b79700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f974408d3e0 0x7f974408d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43856/0 (socket says 172.21.15.38:43856) 2020-03-19T23:11:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.060+0000 7f9749b79700 1 -- 172.21.15.38:0/3631599614 learned_addr learned my addr 172.21.15.38:0/3631599614 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 -- 172.21.15.38:0/3631599614 >> v1:172.21.15.38:6789/0 conn(0x7f974408d3e0 legacy=0x7f974408d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.069 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 -- 172.21.15.38:0/3631599614 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744060810 con 0x7f974408d000 2020-03-19T23:11:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f9749378700 1 -- 172.21.15.38:0/3631599614 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574599059 0 0) 0x7f97440605e0 con 0x7f974408d3e0 2020-03-19T23:11:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 --2- 172.21.15.38:0/3631599614 >> v2:172.21.15.38:3300/0 conn(0x7f974408d000 0x7f97440907c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f9738002f30 tx=0x7f9738007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 --2- 172.21.15.38:0/3631599614 >> v2:172.21.15.38:3300/0 conn(0x7f974408d000 0x7f97440907c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f9738002f30 tx=0x7f9738007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 --2- 172.21.15.38:0/3631599614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d000 0x7f97440907c0 secure :-1 s=READY pgs=91 cs=0 l=1 rx=0x7f9738002f30 tx=0x7f9738007a40).ready entity=mon.0 client_cookie=e05cf1a06a87b8d server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 --2- 172.21.15.38:0/3631599614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d000 0x7f97440907c0 secure :-1 s=THROTTLE_DONE pgs=91 cs=0 l=1 rx=0x7f9738002f30 tx=0x7f9738007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f9749378700 1 -- 172.21.15.38:0/3631599614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f97380113f0 con 0x7f974408d000 2020-03-19T23:11:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 --2- 172.21.15.38:0/3631599614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d000 0x7f97440907c0 secure :-1 s=THROTTLE_DONE pgs=91 cs=0 l=1 rx=0x7f9738002f30 tx=0x7f9738007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f9749378700 1 -- 172.21.15.38:0/3631599614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738011550 con 0x7f974408d000 2020-03-19T23:11:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974a37a700 1 --2- 172.21.15.38:0/3631599614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d000 0x7f97440907c0 secure :-1 s=THROTTLE_DONE pgs=91 cs=0 l=1 rx=0x7f9738002f30 tx=0x7f9738007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f9749378700 1 -- 172.21.15.38:0/3631599614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f97380117f0 con 0x7f974408d000 2020-03-19T23:11:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974c5de700 1 -- 172.21.15.38:0/3631599614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d000 msgr2=0x7f97440907c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974c5de700 1 --2- 172.21.15.38:0/3631599614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d000 0x7f97440907c0 secure :-1 s=READY pgs=91 cs=0 l=1 rx=0x7f9738002f30 tx=0x7f9738007a40).stop 2020-03-19T23:11:05.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974c5de700 1 -- 172.21.15.38:0/3631599614 shutdown_connections 2020-03-19T23:11:05.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974c5de700 1 --2- 172.21.15.38:0/3631599614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d000 0x7f97440907c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:05.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974c5de700 1 -- 172.21.15.38:0/3631599614 >> 172.21.15.38:0/3631599614 conn(0x7f9744091540 msgr2=0x7f9744091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:05.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974c5de700 1 -- 172.21.15.38:0/3631599614 shutdown_connections 2020-03-19T23:11:05.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.061+0000 7f974c5de700 1 -- 172.21.15.38:0/3631599614 wait complete. 2020-03-19T23:11:05.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.062+0000 7f974c5de700 1 Processor -- start 2020-03-19T23:11:05.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.062+0000 7f974c5de700 1 -- start start 2020-03-19T23:11:05.073 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f974c5de700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f974408d3e0 0x7f9744122c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:05.073 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f974c5de700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9744121350 con 0x7f974408d000 2020-03-19T23:11:05.073 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f974c5de700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974411f010 con 0x7f974408d3e0 2020-03-19T23:11:05.073 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f974a37a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f974408d000 0x7f9744122560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43858/0 (socket says 172.21.15.38:43858) 2020-03-19T23:11:05.074 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f974a37a700 1 -- 172.21.15.38:0/740180173 learned_addr learned my addr 172.21.15.38:0/740180173 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:05.074 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> v2:172.21.15.38:3300/0 conn(0x7f974408d3e0 0x7f9744122c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:05.074 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088573826 0 0) 0x7f9744121350 con 0x7f974408d000 2020-03-19T23:11:05.074 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9724002fe0 con 0x7f974408d000 2020-03-19T23:11:05.074 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 -- 172.21.15.38:0/740180173 >> v1:172.21.15.38:6789/0 conn(0x7f974408d000 legacy=0x7f9744122560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 -- 172.21.15.38:0/740180173 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744121040 con 0x7f974408d3e0 2020-03-19T23:11:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> v2:172.21.15.38:3300/0 conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> v2:172.21.15.38:3300/0 conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=READY pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:05.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9740005bc0 con 0x7f974408d3e0 2020-03-19T23:11:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740005d20 con 0x7f974408d3e0 2020-03-19T23:11:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974411eb90 con 0x7f974408d3e0 2020-03-19T23:11:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f97400060c0 con 0x7f974408d3e0 2020-03-19T23:11:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.063+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974411fe20 con 0x7f974408d3e0 2020-03-19T23:11:05.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f974000a320 con 0x7f974408d3e0 2020-03-19T23:11:05.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f97377fe700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 0x7f972402ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:05.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f9740034140 con 0x7f974408d3e0 2020-03-19T23:11:05.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f974a37a700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 0x7f972402ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:05.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f974a37a700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 0x7f972402ac30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9738000b80 tx=0x7f9738007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f974a37a700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 0x7f972402ac30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9738000b80 tx=0x7f9738007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f974a37a700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 0x7f972402ac30 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7f9738000b80 tx=0x7f9738007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:05.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.064+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744056db0 con 0x7f974408d3e0 2020-03-19T23:11:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.068+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.068+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f9740007520 con 0x7f974408d3e0 2020-03-19T23:11:05.217 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.216+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f974406aaf0 con 0x7f974408d3e0 2020-03-19T23:11:05.218 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.216+0000 7f9749b79700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=THROTTLE_DONE pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.218 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.216+0000 7f97377fe700 1 -- 172.21.15.38:0/740180173 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8814 (secure 0 0 0) 0x7f9740011d00 con 0x7f974408d3e0 2020-03-19T23:11:05.218 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:05.218 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":15,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:04.634342+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 msgr2=0x7f972402ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 0x7f972402ac30 secure :-1 s=READY pgs=24 cs=0 l=1 rx=0x7f9738000b80 tx=0x7f9738007a40).stop 2020-03-19T23:11:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 msgr2=0x7f9744122c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 secure :-1 s=READY pgs=93 cs=0 l=1 rx=0x7f9740000b80 tx=0x7f9740010860).stop 2020-03-19T23:11:05.220 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 shutdown_connections 2020-03-19T23:11:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9724028770 0x7f972402ac30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 --2- 172.21.15.38:0/740180173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974408d3e0 0x7f9744122c70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 >> 172.21.15.38:0/740180173 conn(0x7f9744091540 msgr2=0x7f97440629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 shutdown_connections 2020-03-19T23:11:05.221 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.218+0000 7f974c5de700 1 -- 172.21.15.38:0/740180173 wait complete. 2020-03-19T23:11:05.230 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs set cephfs standby_count_wanted 0 2020-03-19T23:11:05.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.345+0000 7f511e46f700 1 Processor -- start 2020-03-19T23:11:05.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.345+0000 7f511e46f700 1 -- start start 2020-03-19T23:11:05.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f511e46f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f5118122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:05.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f511e46f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51181228a0 con 0x7f5118122f40 2020-03-19T23:11:05.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f511e46f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5118092200 con 0x7f51181224c0 2020-03-19T23:11:05.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f5118122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:05.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f51177fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5118122f40 0x7f5118060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43866/0 (socket says 172.21.15.38:43866) 2020-03-19T23:11:05.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f51177fe700 1 -- 172.21.15.38:0/1543799017 learned_addr learned my addr 172.21.15.38:0/1543799017 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:05.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 -- 172.21.15.38:0/1543799017 >> v1:172.21.15.38:6789/0 conn(0x7f5118122f40 legacy=0x7f5118060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 -- 172.21.15.38:0/1543799017 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118090b60 con 0x7f51181224c0 2020-03-19T23:11:05.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 --2- 172.21.15.38:0/1543799017 >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f5118122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 --2- 172.21.15.38:0/1543799017 >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f5118122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 --2- 172.21.15.38:0/1543799017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f5118122ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).ready entity=mon.0 client_cookie=7dba16e71184dea1 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:05.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 --2- 172.21.15.38:0/1543799017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f5118122ae0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5116ffd700 1 -- 172.21.15.38:0/1543799017 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5108008e80 con 0x7f51181224c0 2020-03-19T23:11:05.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 --2- 172.21.15.38:0/1543799017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f5118122ae0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5116ffd700 1 -- 172.21.15.38:0/1543799017 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51080113f0 con 0x7f51181224c0 2020-03-19T23:11:05.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5117fff700 1 --2- 172.21.15.38:0/1543799017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f5118122ae0 secure :-1 s=THROTTLE_DONE pgs=95 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f5116ffd700 1 -- 172.21.15.38:0/1543799017 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5108011710 con 0x7f51181224c0 2020-03-19T23:11:05.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f511e46f700 1 -- 172.21.15.38:0/1543799017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 msgr2=0x7f5118122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.346+0000 7f511e46f700 1 --2- 172.21.15.38:0/1543799017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f5118122ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).stop 2020-03-19T23:11:05.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.347+0000 7f511e46f700 1 -- 172.21.15.38:0/1543799017 shutdown_connections 2020-03-19T23:11:05.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.347+0000 7f511e46f700 1 --2- 172.21.15.38:0/1543799017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f5118122ae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:05.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.347+0000 7f511e46f700 1 -- 172.21.15.38:0/1543799017 >> 172.21.15.38:0/1543799017 conn(0x7f5118091390 msgr2=0x7f5118091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:05.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.347+0000 7f511e46f700 1 -- 172.21.15.38:0/1543799017 shutdown_connections 2020-03-19T23:11:05.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.347+0000 7f511e46f700 1 -- 172.21.15.38:0/1543799017 wait complete. 2020-03-19T23:11:05.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f511e46f700 1 Processor -- start 2020-03-19T23:11:05.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f511e46f700 1 -- start start 2020-03-19T23:11:05.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f511e46f700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f511811ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:05.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f511e46f700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511810d560 con 0x7f5118122f40 2020-03-19T23:11:05.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f511e46f700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511810e0a0 con 0x7f51181224c0 2020-03-19T23:11:05.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f5117fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f511811ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:05.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f51177fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5118122f40 0x7f511811f0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43870/0 (socket says 172.21.15.38:43870) 2020-03-19T23:11:05.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.348+0000 7f51177fe700 1 -- 172.21.15.38:0/2337766499 learned_addr learned my addr 172.21.15.38:0/2337766499 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:05.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 -- 172.21.15.38:0/2337766499 >> v1:172.21.15.38:6789/0 conn(0x7f5118122f40 legacy=0x7f511811f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:05.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849603424 0 0) 0x7f511810d560 con 0x7f5118122f40 2020-03-19T23:11:05.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 -- 172.21.15.38:0/2337766499 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511810e940 con 0x7f51181224c0 2020-03-19T23:11:05.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> v2:172.21.15.38:3300/0 conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:05.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f510800f740 con 0x7f51181224c0 2020-03-19T23:11:05.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108007150 con 0x7f51181224c0 2020-03-19T23:11:05.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511810e630 con 0x7f51181224c0 2020-03-19T23:11:05.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5108007430 con 0x7f51181224c0 2020-03-19T23:11:05.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511810c4e0 con 0x7f51181224c0 2020-03-19T23:11:05.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.349+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f51080056c0 con 0x7f51181224c0 2020-03-19T23:11:05.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f51157fa700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 0x7f50f802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:05.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f5108034600 con 0x7f51181224c0 2020-03-19T23:11:05.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f51177fe700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 0x7f50f802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:05.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5118090b60 con 0x7f51181224c0 2020-03-19T23:11:05.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f51177fe700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 0x7f50f802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f510c000ef0 tx=0x7f510c0016b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f51177fe700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 0x7f50f802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f510c000ef0 tx=0x7f510c0016b0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.350+0000 7f51177fe700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 0x7f50f802a740 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7f510c000ef0 tx=0x7f510c0016b0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:05.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.354+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:05.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.354+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f5118090b60 con 0x7f51181224c0 2020-03-19T23:11:05.555 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:05.555+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5118090b60 con 0x7f51181224c0 2020-03-19T23:11:06.302 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.301+0000 7f5117fff700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=THROTTLE_DONE pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.302 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.301+0000 7f51157fa700 1 -- 172.21.15.38:0/2337766499 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v3) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5118090b60 con 0x7f51181224c0 2020-03-19T23:11:06.305 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 msgr2=0x7f50f802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.305 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 0x7f50f802a740 secure :-1 s=READY pgs=25 cs=0 l=1 rx=0x7f510c000ef0 tx=0x7f510c0016b0).stop 2020-03-19T23:11:06.305 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 msgr2=0x7f511811ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.305 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 secure :-1 s=READY pgs=97 cs=0 l=1 rx=0x7f5108002f30 tx=0x7f5108007a40).stop 2020-03-19T23:11:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 shutdown_connections 2020-03-19T23:11:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51181224c0 0x7f511811ec40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 --2- 172.21.15.38:0/2337766499 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f50f8028280 0x7f50f802a740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.303+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 >> 172.21.15.38:0/2337766499 conn(0x7f5118091390 msgr2=0x7f5118121920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.304+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 shutdown_connections 2020-03-19T23:11:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.304+0000 7f511e46f700 1 -- 172.21.15.38:0/2337766499 wait complete. 2020-03-19T23:11:06.317 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-03-19T23:11:06.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.436+0000 7f717c3f5700 1 Processor -- start 2020-03-19T23:11:06.437 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.436+0000 7f717c3f5700 1 -- start start 2020-03-19T23:11:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.436+0000 7f717c3f5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f717408daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.436+0000 7f717c3f5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71740605e0 con 0x7f717408d000 2020-03-19T23:11:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.436+0000 7f717c3f5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717408ef70 con 0x7f71740907c0 2020-03-19T23:11:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.436+0000 7f7179990700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f717408daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717a191700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f717408d000 0x7f717408d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43874/0 (socket says 172.21.15.38:43874) 2020-03-19T23:11:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717a191700 1 -- 172.21.15.38:0/1936960440 learned_addr learned my addr 172.21.15.38:0/1936960440 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 -- 172.21.15.38:0/1936960440 >> v1:172.21.15.38:6789/0 conn(0x7f717408d000 legacy=0x7f717408d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 -- 172.21.15.38:0/1936960440 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174060810 con 0x7f71740907c0 2020-03-19T23:11:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717918f700 1 -- 172.21.15.38:0/1936960440 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331888131 0 0) 0x7f71740605e0 con 0x7f717408d000 2020-03-19T23:11:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 --2- 172.21.15.38:0/1936960440 >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f717408daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7174090c70 tx=0x7f71680088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.443 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 --2- 172.21.15.38:0/1936960440 >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f717408daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7174090c70 tx=0x7f71680088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 --2- 172.21.15.38:0/1936960440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f717408daf0 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f71680088e0).ready entity=mon.0 client_cookie=2fb8f1916f16cf0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:06.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 --2- 172.21.15.38:0/1936960440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f717408daf0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f71680088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717918f700 1 -- 172.21.15.38:0/1936960440 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f71680113f0 con 0x7f71740907c0 2020-03-19T23:11:06.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 --2- 172.21.15.38:0/1936960440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f717408daf0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f71680088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f7179990700 1 --2- 172.21.15.38:0/1936960440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f717408daf0 secure :-1 s=THROTTLE_DONE pgs=99 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f71680088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717918f700 1 -- 172.21.15.38:0/1936960440 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7168011550 con 0x7f71740907c0 2020-03-19T23:11:06.444 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717918f700 1 -- 172.21.15.38:0/1936960440 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7168011830 con 0x7f71740907c0 2020-03-19T23:11:06.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717c3f5700 1 -- 172.21.15.38:0/1936960440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 msgr2=0x7f717408daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.437+0000 7f717c3f5700 1 --2- 172.21.15.38:0/1936960440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f717408daf0 secure :-1 s=READY pgs=99 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f71680088e0).stop 2020-03-19T23:11:06.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.438+0000 7f717c3f5700 1 -- 172.21.15.38:0/1936960440 shutdown_connections 2020-03-19T23:11:06.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.438+0000 7f717c3f5700 1 --2- 172.21.15.38:0/1936960440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f717408daf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.445 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.438+0000 7f717c3f5700 1 -- 172.21.15.38:0/1936960440 >> 172.21.15.38:0/1936960440 conn(0x7f7174091540 msgr2=0x7f7174091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:06.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.438+0000 7f717c3f5700 1 -- 172.21.15.38:0/1936960440 shutdown_connections 2020-03-19T23:11:06.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.438+0000 7f717c3f5700 1 -- 172.21.15.38:0/1936960440 wait complete. 2020-03-19T23:11:06.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.438+0000 7f717c3f5700 1 Processor -- start 2020-03-19T23:11:06.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.438+0000 7f717c3f5700 1 -- start start 2020-03-19T23:11:06.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f717c3f5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f71741228e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:06.446 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f717c3f5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7174120d20 con 0x7f717408d000 2020-03-19T23:11:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f717c3f5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7174122d40 con 0x7f71740907c0 2020-03-19T23:11:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f717a191700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f717408d000 0x7f71741221d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43876/0 (socket says 172.21.15.38:43876) 2020-03-19T23:11:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f717a191700 1 -- 172.21.15.38:0/1694751378 learned_addr learned my addr 172.21.15.38:0/1694751378 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f71741228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513488358 0 0) 0x7f7174120d20 con 0x7f717408d000 2020-03-19T23:11:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7154002fe0 con 0x7f717408d000 2020-03-19T23:11:06.447 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 -- 172.21.15.38:0/1694751378 >> v1:172.21.15.38:6789/0 conn(0x7f717408d000 legacy=0x7f71741221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 -- 172.21.15.38:0/1694751378 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174120fc0 con 0x7f71740907c0 2020-03-19T23:11:06.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> v2:172.21.15.38:3300/0 conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=READY pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:06.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.448 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f716800f6c0 con 0x7f71740907c0 2020-03-19T23:11:06.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.439+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800f820 con 0x7f71740907c0 2020-03-19T23:11:06.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f71680057f0 con 0x7f71740907c0 2020-03-19T23:11:06.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717411fc40 con 0x7f71740907c0 2020-03-19T23:11:06.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717411f080 con 0x7f71740907c0 2020-03-19T23:11:06.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f7168005a60 con 0x7f71740907c0 2020-03-19T23:11:06.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f71737fe700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 0x7f715402ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:06.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.440+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f7168034520 con 0x7f71740907c0 2020-03-19T23:11:06.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.441+0000 7f717a191700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 0x7f715402ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:06.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.441+0000 7f717a191700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 0x7f715402ac20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f716c0058a0 tx=0x7f716c001a20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.441+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717411fa50 con 0x7f71740907c0 2020-03-19T23:11:06.450 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.441+0000 7f717a191700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 0x7f715402ac20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f716c0058a0 tx=0x7f716c001a20).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.441+0000 7f717a191700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 0x7f715402ac20 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f716c0058a0 tx=0x7f716c001a20).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:06.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.444+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.451 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.444+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f71680132e0 con 0x7f71740907c0 2020-03-19T23:11:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.638+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7174056db0 con 0x7f71740907c0 2020-03-19T23:11:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.638+0000 7f7179990700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=THROTTLE_DONE pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.638+0000 7f71737fe700 1 -- 172.21.15.38:0/1694751378 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1420 (secure 0 0 0) 0x7f7168009dc0 con 0x7f71740907c0 2020-03-19T23:11:06.639 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:06.640 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":3,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-03-19T23:11:04.634508+0000","modified":"2020-03-19T23:11:06.296369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-03-19T23:11:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 msgr2=0x7f715402ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 0x7f715402ac20 secure :-1 s=READY pgs=26 cs=0 l=1 rx=0x7f716c0058a0 tx=0x7f716c001a20).stop 2020-03-19T23:11:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 msgr2=0x7f71741228e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 secure :-1 s=READY pgs=101 cs=0 l=1 rx=0x7f7174090c70 tx=0x7f7174067850).stop 2020-03-19T23:11:06.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 shutdown_connections 2020-03-19T23:11:06.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7154028760 0x7f715402ac20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 --2- 172.21.15.38:0/1694751378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71740907c0 0x7f71741228e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 >> 172.21.15.38:0/1694751378 conn(0x7f7174091540 msgr2=0x7f71740629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:06.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 shutdown_connections 2020-03-19T23:11:06.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.639+0000 7f717c3f5700 1 -- 172.21.15.38:0/1694751378 wait complete. 2020-03-19T23:11:06.642 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 3 2020-03-19T23:11:06.651 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:06.763 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.762+0000 7fb63c1f5700 1 Processor -- start 2020-03-19T23:11:06.763 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.762+0000 7fb63c1f5700 1 -- start start 2020-03-19T23:11:06.769 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb63c1f5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb634066760 0x7fb634066b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:06.769 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb63c1f5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340912a0 con 0x7fb634090890 2020-03-19T23:11:06.769 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb63c1f5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63408ddb0 con 0x7fb634066760 2020-03-19T23:11:06.769 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639f91700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb634066760 0x7fb634066b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:06.769 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639790700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb634090890 0x7fb63408bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43882/0 (socket says 172.21.15.38:43882) 2020-03-19T23:11:06.770 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639f91700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb634066760 0x7fb634066b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60742/0 (socket says 172.21.15.38:60742) 2020-03-19T23:11:06.770 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639790700 1 -- 172.21.15.38:0/1216945226 learned_addr learned my addr 172.21.15.38:0/1216945226 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:06.770 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639f91700 1 -- 172.21.15.38:0/1216945226 >> v1:172.21.15.38:6789/0 conn(0x7fb634090890 legacy=0x7fb63408bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.770 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639f91700 1 -- 172.21.15.38:0/1216945226 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63408d1b0 con 0x7fb634066760 2020-03-19T23:11:06.770 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639f91700 1 --2- 172.21.15.38:0/1216945226 >> v2:172.21.15.38:3300/0 conn(0x7fb634066760 0x7fb634066b40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fb6280032b0 tx=0x7fb628007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.763+0000 7fb639f91700 1 --2- 172.21.15.38:0/1216945226 >> v2:172.21.15.38:3300/0 conn(0x7fb634066760 0x7fb634066b40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fb6280032b0 tx=0x7fb628007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb639f91700 1 --2- 172.21.15.38:0/1216945226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634066760 0x7fb634066b40 secure :-1 s=READY pgs=103 cs=0 l=1 rx=0x7fb6280032b0 tx=0x7fb628007dc0).ready entity=mon.0 client_cookie=404c0f6581deac64 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb639f91700 1 --2- 172.21.15.38:0/1216945226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634066760 0x7fb634066b40 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fb6280032b0 tx=0x7fb628007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb638f8f700 1 -- 172.21.15.38:0/1216945226 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb628011cd0 con 0x7fb634066760 2020-03-19T23:11:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb639f91700 1 --2- 172.21.15.38:0/1216945226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634066760 0x7fb634066b40 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fb6280032b0 tx=0x7fb628007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.771 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb639f91700 1 --2- 172.21.15.38:0/1216945226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634066760 0x7fb634066b40 secure :-1 s=THROTTLE_DONE pgs=103 cs=0 l=1 rx=0x7fb6280032b0 tx=0x7fb628007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb638f8f700 1 -- 172.21.15.38:0/1216945226 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb628011e30 con 0x7fb634066760 2020-03-19T23:11:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb638f8f700 1 -- 172.21.15.38:0/1216945226 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb62800a290 con 0x7fb634066760 2020-03-19T23:11:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb63c1f5700 1 -- 172.21.15.38:0/1216945226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634066760 msgr2=0x7fb634066b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb63c1f5700 1 --2- 172.21.15.38:0/1216945226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634066760 0x7fb634066b40 secure :-1 s=READY pgs=103 cs=0 l=1 rx=0x7fb6280032b0 tx=0x7fb628007dc0).stop 2020-03-19T23:11:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb63c1f5700 1 -- 172.21.15.38:0/1216945226 shutdown_connections 2020-03-19T23:11:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb63c1f5700 1 --2- 172.21.15.38:0/1216945226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634066760 0x7fb634066b40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.772 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb63c1f5700 1 -- 172.21.15.38:0/1216945226 >> 172.21.15.38:0/1216945226 conn(0x7fb634091540 msgr2=0x7fb634091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb63c1f5700 1 -- 172.21.15.38:0/1216945226 shutdown_connections 2020-03-19T23:11:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.764+0000 7fb63c1f5700 1 -- 172.21.15.38:0/1216945226 wait complete. 2020-03-19T23:11:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.765+0000 7fb63c1f5700 1 Processor -- start 2020-03-19T23:11:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.765+0000 7fb63c1f5700 1 -- start start 2020-03-19T23:11:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.765+0000 7fb63c1f5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb634090890 0x7fb634061480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.765+0000 7fb63c1f5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb634120d60 con 0x7fb634066760 2020-03-19T23:11:06.773 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.765+0000 7fb63c1f5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb634121fc0 con 0x7fb634090890 2020-03-19T23:11:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.765+0000 7fb639f91700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb634066760 0x7fb634060d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43886/0 (socket says 172.21.15.38:43886) 2020-03-19T23:11:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.765+0000 7fb639f91700 1 -- 172.21.15.38:0/2210585963 learned_addr learned my addr 172.21.15.38:0/2210585963 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> v2:172.21.15.38:3300/0 conn(0x7fb634090890 0x7fb634061480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358291682 0 0) 0x7fb634120d60 con 0x7fb634066760 2020-03-19T23:11:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb614002fe0 con 0x7fb634066760 2020-03-19T23:11:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 -- 172.21.15.38:0/2210585963 >> v1:172.21.15.38:6789/0 conn(0x7fb634066760 legacy=0x7fb634060d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.774 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 -- 172.21.15.38:0/2210585963 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634120490 con 0x7fb634090890 2020-03-19T23:11:06.776 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> v2:172.21.15.38:3300/0 conn(0x7fb634090890 0x7fb634061480 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.776 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> v2:172.21.15.38:3300/0 conn(0x7fb634090890 0x7fb634061480 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.776 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=READY pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:06.776 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb630005b20 con 0x7fb634090890 2020-03-19T23:11:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630011e90 con 0x7fb634090890 2020-03-19T23:11:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb634061ef0 con 0x7fb634090890 2020-03-19T23:11:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb630005db0 con 0x7fb634090890 2020-03-19T23:11:06.777 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.766+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb634124bb0 con 0x7fb634090890 2020-03-19T23:11:06.778 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.778 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fb63000a320 con 0x7fb634090890 2020-03-19T23:11:06.778 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.778 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb626ffd700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 0x7fb61402abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:06.778 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fb630033f30 con 0x7fb634090890 2020-03-19T23:11:06.778 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb639f91700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 0x7fb61402abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:06.778 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb639f91700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 0x7fb61402abe0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb628008c60 tx=0x7fb628007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.779 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb634120680 con 0x7fb634090890 2020-03-19T23:11:06.779 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb639f91700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 0x7fb61402abe0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb628008c60 tx=0x7fb628007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.779 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.767+0000 7fb639f91700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 0x7fb61402abe0 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7fb628008c60 tx=0x7fb628007dc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:06.779 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.771+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.780 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.771+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fb6300090b0 con 0x7fb634090890 2020-03-19T23:11:06.920 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.919+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb634056db0 con 0x7fb634090890 2020-03-19T23:11:06.921 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.920+0000 7fb639790700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=THROTTLE_DONE pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:06.921 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.920+0000 7fb626ffd700 1 -- 172.21.15.38:0/2210585963 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8801 (secure 0 0 0) 0x7fb630011c90 con 0x7fb634090890 2020-03-19T23:11:06.922 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:06.922 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:05.639978+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:06.923 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 msgr2=0x7fb61402abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.923 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 0x7fb61402abe0 secure :-1 s=READY pgs=27 cs=0 l=1 rx=0x7fb628008c60 tx=0x7fb628007dc0).stop 2020-03-19T23:11:06.923 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 msgr2=0x7fb634061480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:06.923 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 secure :-1 s=READY pgs=105 cs=0 l=1 rx=0x7fb634121540 tx=0x7fb630010830).stop 2020-03-19T23:11:06.923 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 shutdown_connections 2020-03-19T23:11:06.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb614028720 0x7fb61402abe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 --2- 172.21.15.38:0/2210585963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb634090890 0x7fb634061480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:06.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 >> 172.21.15.38:0/2210585963 conn(0x7fb634091540 msgr2=0x7fb634124db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:06.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 shutdown_connections 2020-03-19T23:11:06.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:06.921+0000 7fb63c1f5700 1 -- 172.21.15.38:0/2210585963 wait complete. 2020-03-19T23:11:06.933 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-03-19T23:11:06.934 INFO:tasks.ceph.mds.a:Restarting daemon 2020-03-19T23:11:06.934 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-03-19T23:11:06.936 INFO:tasks.ceph.mds.a:Started 2020-03-19T23:11:06.937 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:06.937 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-03-19T23:11:07.016 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2020-03-19T23:11:07.051 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.050+0000 7fdb2d081700 1 Processor -- start 2020-03-19T23:11:07.051 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.050+0000 7fdb2d081700 1 -- start start 2020-03-19T23:11:07.058 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2d081700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb28125eb0 0x7fdb2812b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:07.058 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2d081700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2808f2f0 con 0x7fdb28096640 2020-03-19T23:11:07.058 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2d081700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb28090000 con 0x7fdb28125eb0 2020-03-19T23:11:07.058 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb28125eb0 0x7fdb2812b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.058 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb28125eb0 0x7fdb2812b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60762/0 (socket says 172.21.15.38:60762) 2020-03-19T23:11:07.058 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 -- 172.21.15.38:0/347924741 learned_addr learned my addr 172.21.15.38:0/347924741 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb26d9d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdb28096640 0x7fdb28126740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43900/0 (socket says 172.21.15.38:43900) 2020-03-19T23:11:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 -- 172.21.15.38:0/347924741 >> v1:172.21.15.38:6789/0 conn(0x7fdb28096640 legacy=0x7fdb28126740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb25d9b700 1 -- 172.21.15.38:0/347924741 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807019394 0 0) 0x7fdb2808f2f0 con 0x7fdb28096640 2020-03-19T23:11:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 -- 172.21.15.38:0/347924741 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28096a20 con 0x7fdb28125eb0 2020-03-19T23:11:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- 172.21.15.38:0/347924741 >> v2:172.21.15.38:3300/0 conn(0x7fdb28125eb0 0x7fdb2812b240 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fdb28122930 tx=0x7fdb1c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- 172.21.15.38:0/347924741 >> v2:172.21.15.38:3300/0 conn(0x7fdb28125eb0 0x7fdb2812b240 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fdb28122930 tx=0x7fdb1c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.059 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- 172.21.15.38:0/347924741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28125eb0 0x7fdb2812b240 secure :-1 s=READY pgs=111 cs=0 l=1 rx=0x7fdb28122930 tx=0x7fdb1c0088e0).ready entity=mon.0 client_cookie=12eb4c8be0301987 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- 172.21.15.38:0/347924741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28125eb0 0x7fdb2812b240 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7fdb28122930 tx=0x7fdb1c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb25d9b700 1 -- 172.21.15.38:0/347924741 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb1c0114b0 con 0x7fdb28125eb0 2020-03-19T23:11:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- 172.21.15.38:0/347924741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28125eb0 0x7fdb2812b240 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7fdb28122930 tx=0x7fdb1c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb25d9b700 1 -- 172.21.15.38:0/347924741 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c011610 con 0x7fdb28125eb0 2020-03-19T23:11:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2659c700 1 --2- 172.21.15.38:0/347924741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28125eb0 0x7fdb2812b240 secure :-1 s=THROTTLE_DONE pgs=111 cs=0 l=1 rx=0x7fdb28122930 tx=0x7fdb1c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb25d9b700 1 -- 172.21.15.38:0/347924741 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb1c0118b0 con 0x7fdb28125eb0 2020-03-19T23:11:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.051+0000 7fdb2d081700 1 -- 172.21.15.38:0/347924741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28125eb0 msgr2=0x7fdb2812b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.052+0000 7fdb2d081700 1 --2- 172.21.15.38:0/347924741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28125eb0 0x7fdb2812b240 secure :-1 s=READY pgs=111 cs=0 l=1 rx=0x7fdb28122930 tx=0x7fdb1c0088e0).stop 2020-03-19T23:11:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.052+0000 7fdb2d081700 1 -- 172.21.15.38:0/347924741 shutdown_connections 2020-03-19T23:11:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.052+0000 7fdb2d081700 1 --2- 172.21.15.38:0/347924741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28125eb0 0x7fdb2812b240 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.052+0000 7fdb2d081700 1 -- 172.21.15.38:0/347924741 >> 172.21.15.38:0/347924741 conn(0x7fdb2805bf00 msgr2=0x7fdb280990a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.052+0000 7fdb2d081700 1 -- 172.21.15.38:0/347924741 shutdown_connections 2020-03-19T23:11:07.061 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.052+0000 7fdb2d081700 1 -- 172.21.15.38:0/347924741 wait complete. 2020-03-19T23:11:07.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb2d081700 1 Processor -- start 2020-03-19T23:11:07.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb2d081700 1 -- start start 2020-03-19T23:11:07.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb2d081700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb28096640 0x7fdb28092e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb2d081700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb28096260 con 0x7fdb28125eb0 2020-03-19T23:11:07.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb2d081700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb28092720 con 0x7fdb28096640 2020-03-19T23:11:07.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb26d9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb28096640 0x7fdb28092e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb2659c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdb28125eb0 0x7fdb28093280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43906/0 (socket says 172.21.15.38:43906) 2020-03-19T23:11:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.053+0000 7fdb2659c700 1 -- 172.21.15.38:0/3499543196 learned_addr learned my addr 172.21.15.38:0/3499543196 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 -- 172.21.15.38:0/3499543196 >> v1:172.21.15.38:6789/0 conn(0x7fdb28125eb0 legacy=0x7fdb28093280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448697664 0 0) 0x7fdb28096260 con 0x7fdb28125eb0 2020-03-19T23:11:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 -- 172.21.15.38:0/3499543196 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28120410 con 0x7fdb28096640 2020-03-19T23:11:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> v2:172.21.15.38:3300/0 conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> v2:172.21.15.38:3300/0 conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=READY pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb20004ad0 con 0x7fdb28096640 2020-03-19T23:11:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb20004c30 con 0x7fdb28096640 2020-03-19T23:11:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb28093fd0 con 0x7fdb28096640 2020-03-19T23:11:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb200125f0 con 0x7fdb28096640 2020-03-19T23:11:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb28094e50 con 0x7fdb28096640 2020-03-19T23:11:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fdb200128b0 con 0x7fdb28096640 2020-03-19T23:11:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.054+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.055+0000 7fdb17fff700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 0x7fdb0402a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.055+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fdb200345f0 con 0x7fdb28096640 2020-03-19T23:11:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.055+0000 7fdb2659c700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 0x7fdb0402a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.055+0000 7fdb2659c700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 0x7fdb0402a670 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fdb1c010280 tx=0x7fdb28096bb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.055+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb280955a0 con 0x7fdb28096640 2020-03-19T23:11:07.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.055+0000 7fdb2659c700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 0x7fdb0402a670 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fdb1c010280 tx=0x7fdb28096bb0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.067 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.055+0000 7fdb2659c700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 0x7fdb0402a670 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7fdb1c010280 tx=0x7fdb28096bb0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.067 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.058+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.067 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.058+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fdb20012b40 con 0x7fdb28096640 2020-03-19T23:11:07.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.206+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb28056db0 con 0x7fdb28096640 2020-03-19T23:11:07.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.206+0000 7fdb26d9d700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=THROTTLE_DONE pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.206+0000 7fdb17fff700 1 -- 172.21.15.38:0/3499543196 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8801 (secure 0 0 0) 0x7fdb200068b0 con 0x7fdb28096640 2020-03-19T23:11:07.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 msgr2=0x7fdb0402a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 0x7fdb0402a670 secure :-1 s=READY pgs=28 cs=0 l=1 rx=0x7fdb1c010280 tx=0x7fdb28096bb0).stop 2020-03-19T23:11:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 msgr2=0x7fdb28092e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 secure :-1 s=READY pgs=113 cs=0 l=1 rx=0x7fdb20000f00 tx=0x7fdb20010c50).stop 2020-03-19T23:11:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 shutdown_connections 2020-03-19T23:11:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb28096640 0x7fdb28092e20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 --2- 172.21.15.38:0/3499543196 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdb040281b0 0x7fdb0402a670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 >> 172.21.15.38:0/3499543196 conn(0x7fdb2805bf00 msgr2=0x7fdb28122970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 shutdown_connections 2020-03-19T23:11:07.214 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.208+0000 7fdb2d081700 1 -- 172.21.15.38:0/3499543196 wait complete. 2020-03-19T23:11:07.214 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:07.215 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:05.639978+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:07.219 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:10:59.540840+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'12', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:11:02.421156+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_metadata', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'metadata': u'cephfs'}}, u'options': {u'pg_autoscale_bias': 4, u'recovery_priority': 5, u'pg_num_min': 16}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'15', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:11:03.595719+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_data', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 3, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'data': u'cephfs'}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'16', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T23:11:07.220 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T23:11:07.346 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.345+0000 7f47a8451700 1 Processor -- start 2020-03-19T23:11:07.346 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.345+0000 7f47a8451700 1 -- start start 2020-03-19T23:11:07.351 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.345+0000 7f47a8451700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a0127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.345+0000 7f47a8451700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a0060f10 con 0x7f47a0065970 2020-03-19T23:11:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.345+0000 7f47a8451700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a0061490 con 0x7f47a0125060 2020-03-19T23:11:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.345+0000 7f47a59ec700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a0127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a61ed700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f47a0065970 0x7f47a0065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43918/0 (socket says 172.21.15.38:43918) 2020-03-19T23:11:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a0127450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60780/0 (socket says 172.21.15.38:60780) 2020-03-19T23:11:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a61ed700 1 -- 172.21.15.38:0/4105263396 learned_addr learned my addr 172.21.15.38:0/4105263396 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:07.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 -- 172.21.15.38:0/4105263396 >> v1:172.21.15.38:6789/0 conn(0x7f47a0065970 legacy=0x7f47a0065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a51eb700 1 -- 172.21.15.38:0/4105263396 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433771997 0 0) 0x7f479c0010d0 con 0x7f47a0065970 2020-03-19T23:11:07.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 -- 172.21.15.38:0/4105263396 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a0061730 con 0x7f47a0125060 2020-03-19T23:11:07.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 --2- 172.21.15.38:0/4105263396 >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a0127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 --2- 172.21.15.38:0/4105263396 >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a0127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 --2- 172.21.15.38:0/4105263396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a0127450 secure :-1 s=READY pgs=115 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).ready entity=mon.0 client_cookie=4574e6dc65ee9d6f server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.353 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 --2- 172.21.15.38:0/4105263396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a0127450 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a51eb700 1 -- 172.21.15.38:0/4105263396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4790013600 con 0x7f47a0125060 2020-03-19T23:11:07.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 --2- 172.21.15.38:0/4105263396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a0127450 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a59ec700 1 --2- 172.21.15.38:0/4105263396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a0127450 secure :-1 s=THROTTLE_DONE pgs=115 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a51eb700 1 -- 172.21.15.38:0/4105263396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790013760 con 0x7f47a0125060 2020-03-19T23:11:07.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a51eb700 1 -- 172.21.15.38:0/4105263396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4790013a40 con 0x7f47a0125060 2020-03-19T23:11:07.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a8451700 1 -- 172.21.15.38:0/4105263396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 msgr2=0x7f47a0127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.346+0000 7f47a8451700 1 --2- 172.21.15.38:0/4105263396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a0127450 secure :-1 s=READY pgs=115 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).stop 2020-03-19T23:11:07.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.347+0000 7f47a8451700 1 -- 172.21.15.38:0/4105263396 shutdown_connections 2020-03-19T23:11:07.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.347+0000 7f47a8451700 1 --2- 172.21.15.38:0/4105263396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a0127450 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.347+0000 7f47a8451700 1 -- 172.21.15.38:0/4105263396 >> 172.21.15.38:0/4105263396 conn(0x7f47a0091390 msgr2=0x7f47a0091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:07.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.347+0000 7f47a8451700 1 -- 172.21.15.38:0/4105263396 shutdown_connections 2020-03-19T23:11:07.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.347+0000 7f47a8451700 1 -- 172.21.15.38:0/4105263396 wait complete. 2020-03-19T23:11:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.347+0000 7f47a8451700 1 Processor -- start 2020-03-19T23:11:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.347+0000 7f47a8451700 1 -- start start 2020-03-19T23:11:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a8451700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a012f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a8451700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a010da70 con 0x7f47a0065970 2020-03-19T23:11:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a8451700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a01115e0 con 0x7f47a0125060 2020-03-19T23:11:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a61ed700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f47a0065970 0x7f47a012ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43922/0 (socket says 172.21.15.38:43922) 2020-03-19T23:11:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a61ed700 1 -- 172.21.15.38:0/16396026 learned_addr learned my addr 172.21.15.38:0/16396026 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:07.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a012f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a59ec700 1 -- 172.21.15.38:0/16396026 >> v1:172.21.15.38:6789/0 conn(0x7f47a0065970 legacy=0x7f47a012ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536685675 0 0) 0x7f47a010da70 con 0x7f47a0065970 2020-03-19T23:11:07.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a59ec700 1 -- 172.21.15.38:0/16396026 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a012c990 con 0x7f47a0125060 2020-03-19T23:11:07.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> v2:172.21.15.38:3300/0 conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.348+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4790007150 con 0x7f47a0125060 2020-03-19T23:11:07.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47900072b0 con 0x7f47a0125060 2020-03-19T23:11:07.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a010f250 con 0x7f47a0125060 2020-03-19T23:11:07.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4790007570 con 0x7f47a0125060 2020-03-19T23:11:07.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a010fe90 con 0x7f47a0125060 2020-03-19T23:11:07.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f47900056c0 con 0x7f47a0125060 2020-03-19T23:11:07.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f478f7fe700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 0x7f478002a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f4790036bf0 con 0x7f47a0125060 2020-03-19T23:11:07.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.349+0000 7f47a61ed700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 0x7f478002a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.350+0000 7f47a61ed700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 0x7f478002a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f479c005500 tx=0x7f479c0016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.350+0000 7f47a61ed700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 0x7f478002a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f479c005500 tx=0x7f479c0016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.350+0000 7f47a61ed700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 0x7f478002a6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f479c005500 tx=0x7f479c0016e0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.350+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a0056db0 con 0x7f47a0125060 2020-03-19T23:11:07.380 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.353+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.380 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.353+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f479000b7d0 con 0x7f47a0125060 2020-03-19T23:11:07.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.500+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f47a0067160 con 0x7f47a0125060 2020-03-19T23:11:07.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.500+0000 7f47a59ec700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=THROTTLE_DONE pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.500+0000 7f478f7fe700 1 -- 172.21.15.38:0/16396026 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4790016020 con 0x7f47a0125060 2020-03-19T23:11:07.502 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.501+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 msgr2=0x7f478002a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.501+0000 7f47a8451700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 0x7f478002a6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rx=0x7f479c005500 tx=0x7f479c0016e0).stop 2020-03-19T23:11:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 msgr2=0x7f47a012f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rx=0x7f4790002f30 tx=0x7f4790007a40).stop 2020-03-19T23:11:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 shutdown_connections 2020-03-19T23:11:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4780028200 0x7f478002a6c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 --2- 172.21.15.38:0/16396026 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a0125060 0x7f47a012f2c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 >> 172.21.15.38:0/16396026 conn(0x7f47a0091390 msgr2=0x7f47a008dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 shutdown_connections 2020-03-19T23:11:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.502+0000 7f47a8451700 1 -- 172.21.15.38:0/16396026 wait complete. 2020-03-19T23:11:07.513 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-03-19T23:11:07.628 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.627+0000 7fa90d9a0700 1 Processor -- start 2020-03-19T23:11:07.629 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.627+0000 7fa90d9a0700 1 -- start start 2020-03-19T23:11:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90d9a0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa90808e5c0 0x7fa908122960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90d9a0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa908123270 con 0x7fa908122dc0 2020-03-19T23:11:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90d9a0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9081223b0 con 0x7fa90808e5c0 2020-03-19T23:11:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90c99e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa90808e5c0 0x7fa908122960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa907fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa908122dc0 0x7fa908060070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43930/0 (socket says 172.21.15.38:43930) 2020-03-19T23:11:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90c99e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa90808e5c0 0x7fa908122960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60788/0 (socket says 172.21.15.38:60788) 2020-03-19T23:11:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa907fff700 1 -- 172.21.15.38:0/2535791253 learned_addr learned my addr 172.21.15.38:0/2535791253 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90c99e700 1 -- 172.21.15.38:0/2535791253 >> v1:172.21.15.38:6789/0 conn(0x7fa908122dc0 legacy=0x7fa908060070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90c99e700 1 -- 172.21.15.38:0/2535791253 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908091f00 con 0x7fa90808e5c0 2020-03-19T23:11:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa9077fe700 1 -- 172.21.15.38:0/2535791253 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577206486 0 0) 0x7fa908123270 con 0x7fa908122dc0 2020-03-19T23:11:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.628+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2535791253 >> v2:172.21.15.38:3300/0 conn(0x7fa90808e5c0 0x7fa908122960 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa8f0002f30 tx=0x7fa8f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2535791253 >> v2:172.21.15.38:3300/0 conn(0x7fa90808e5c0 0x7fa908122960 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa8f0002f30 tx=0x7fa8f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2535791253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa90808e5c0 0x7fa908122960 secure :-1 s=READY pgs=119 cs=0 l=1 rx=0x7fa8f0002f30 tx=0x7fa8f0007a40).ready entity=mon.0 client_cookie=a30b635d752645f6 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2535791253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa90808e5c0 0x7fa908122960 secure :-1 s=THROTTLE_DONE pgs=119 cs=0 l=1 rx=0x7fa8f0002f30 tx=0x7fa8f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa9077fe700 1 -- 172.21.15.38:0/2535791253 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa8f0008e80 con 0x7fa90808e5c0 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2535791253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa90808e5c0 0x7fa908122960 secure :-1 s=THROTTLE_DONE pgs=119 cs=0 l=1 rx=0x7fa8f0002f30 tx=0x7fa8f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2535791253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa90808e5c0 0x7fa908122960 secure :-1 s=THROTTLE_DONE pgs=119 cs=0 l=1 rx=0x7fa8f0002f30 tx=0x7fa8f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa9077fe700 1 -- 172.21.15.38:0/2535791253 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f00113f0 con 0x7fa90808e5c0 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa9077fe700 1 -- 172.21.15.38:0/2535791253 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa8f00116d0 con 0x7fa90808e5c0 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2535791253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa90808e5c0 msgr2=0x7fa908122960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90d9a0700 1 --2- 172.21.15.38:0/2535791253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa90808e5c0 0x7fa908122960 secure :-1 s=READY pgs=119 cs=0 l=1 rx=0x7fa8f0002f30 tx=0x7fa8f0007a40).stop 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2535791253 shutdown_connections 2020-03-19T23:11:07.636 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90d9a0700 1 --2- 172.21.15.38:0/2535791253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa90808e5c0 0x7fa908122960 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.637 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2535791253 >> 172.21.15.38:0/2535791253 conn(0x7fa9080950e0 msgr2=0x7fa908091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:07.637 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2535791253 shutdown_connections 2020-03-19T23:11:07.637 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.629+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2535791253 wait complete. 2020-03-19T23:11:07.637 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.630+0000 7fa90d9a0700 1 Processor -- start 2020-03-19T23:11:07.637 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.630+0000 7fa90d9a0700 1 -- start start 2020-03-19T23:11:07.637 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.630+0000 7fa90d9a0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa908122dc0 0x7fa908132d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.637 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.630+0000 7fa90d9a0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa908062ad0 con 0x7fa90808e5c0 2020-03-19T23:11:07.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa90d9a0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa908062e00 con 0x7fa908122dc0 2020-03-19T23:11:07.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa90c99e700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa90808e5c0 0x7fa908132670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43932/0 (socket says 172.21.15.38:43932) 2020-03-19T23:11:07.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa90c99e700 1 -- 172.21.15.38:0/2595835478 learned_addr learned my addr 172.21.15.38:0/2595835478 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:07.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa908122dc0 0x7fa908132d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300198236 0 0) 0x7fa908062ad0 con 0x7fa90808e5c0 2020-03-19T23:11:07.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8ec002fe0 con 0x7fa90808e5c0 2020-03-19T23:11:07.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 -- 172.21.15.38:0/2595835478 >> v1:172.21.15.38:6789/0 conn(0x7fa90808e5c0 legacy=0x7fa908132670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 -- 172.21.15.38:0/2595835478 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908111360 con 0x7fa908122dc0 2020-03-19T23:11:07.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> v2:172.21.15.38:3300/0 conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> v2:172.21.15.38:3300/0 conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=READY pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa8f8005b20 con 0x7fa908122dc0 2020-03-19T23:11:07.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f8005c80 con 0x7fa908122dc0 2020-03-19T23:11:07.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa908063520 con 0x7fa908122dc0 2020-03-19T23:11:07.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa8f80060c0 con 0x7fa908122dc0 2020-03-19T23:11:07.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.631+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908110770 con 0x7fa908122dc0 2020-03-19T23:11:07.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fa8f800a320 con 0x7fa908122dc0 2020-03-19T23:11:07.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa905ffb700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 0x7fa8ec02ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fa8f80340a0 con 0x7fa908122dc0 2020-03-19T23:11:07.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 0x7fa8ec02ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908056db0 con 0x7fa908122dc0 2020-03-19T23:11:07.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 0x7fa8ec02ac70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa8f00088e0 tx=0x7fa8f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 0x7fa8ec02ac70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa8f00088e0 tx=0x7fa8f0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.644 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.632+0000 7fa90c99e700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 0x7fa8ec02ac70 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7fa8f00088e0 tx=0x7fa8f0007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.644 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.636+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.644 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.636+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fa8f8007700 con 0x7fa908122dc0 2020-03-19T23:11:07.783 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.782+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa90805e820 con 0x7fa908122dc0 2020-03-19T23:11:07.784 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.782+0000 7fa907fff700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=THROTTLE_DONE pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.784 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.782+0000 7fa905ffb700 1 -- 172.21.15.38:0/2595835478 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa8f8011d40 con 0x7fa908122dc0 2020-03-19T23:11:07.784 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:07.785 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.783+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 msgr2=0x7fa8ec02ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.785 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 0x7fa8ec02ac70 secure :-1 s=READY pgs=31 cs=0 l=1 rx=0x7fa8f00088e0 tx=0x7fa8f0007a40).stop 2020-03-19T23:11:07.785 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 msgr2=0x7fa908132d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 secure :-1 s=READY pgs=121 cs=0 l=1 rx=0x7fa8f8000b80 tx=0x7fa8f80108a0).stop 2020-03-19T23:11:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 shutdown_connections 2020-03-19T23:11:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa8ec0287b0 0x7fa8ec02ac70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 --2- 172.21.15.38:0/2595835478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa908122dc0 0x7fa908132d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 >> 172.21.15.38:0/2595835478 conn(0x7fa9080950e0 msgr2=0x7fa9081201e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 shutdown_connections 2020-03-19T23:11:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.784+0000 7fa90d9a0700 1 -- 172.21.15.38:0/2595835478 wait complete. 2020-03-19T23:11:07.795 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-03-19T23:11:07.910 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.909+0000 7fdb0a0c6700 1 Processor -- start 2020-03-19T23:11:07.910 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.909+0000 7fdb0a0c6700 1 -- start start 2020-03-19T23:11:07.913 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb0a0c6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb04127740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:07.913 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb0a0c6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb041223b0 con 0x7fdb0408e5c0 2020-03-19T23:11:07.913 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb0a0c6700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb04091f00 con 0x7fdb04123070 2020-03-19T23:11:07.913 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb088c3700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb04127740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.913 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb090c4700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdb0408e5c0 0x7fdb04122960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43938/0 (socket says 172.21.15.38:43938) 2020-03-19T23:11:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb088c3700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb04127740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60800/0 (socket says 172.21.15.38:60800) 2020-03-19T23:11:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb090c4700 1 -- 172.21.15.38:0/2209407817 learned_addr learned my addr 172.21.15.38:0/2209407817 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb088c3700 1 -- 172.21.15.38:0/2209407817 >> v1:172.21.15.38:6789/0 conn(0x7fdb0408e5c0 legacy=0x7fdb04122960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb088c3700 1 -- 172.21.15.38:0/2209407817 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb04122650 con 0x7fdb04123070 2020-03-19T23:11:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb088c3700 1 --2- 172.21.15.38:0/2209407817 >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb04127740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb088c3700 1 --2- 172.21.15.38:0/2209407817 >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb04127740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.910+0000 7fdb088c3700 1 --2- 172.21.15.38:0/2209407817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb04127740 secure :-1 s=READY pgs=123 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).ready entity=mon.0 client_cookie=328ad1bba576157 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb088c3700 1 --2- 172.21.15.38:0/2209407817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb04127740 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdaf7fff700 1 -- 172.21.15.38:0/2209407817 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb00008bb0 con 0x7fdb04123070 2020-03-19T23:11:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb088c3700 1 --2- 172.21.15.38:0/2209407817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb04127740 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb088c3700 1 --2- 172.21.15.38:0/2209407817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb04127740 secure :-1 s=THROTTLE_DONE pgs=123 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdaf7fff700 1 -- 172.21.15.38:0/2209407817 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb00008d10 con 0x7fdb04123070 2020-03-19T23:11:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdaf7fff700 1 -- 172.21.15.38:0/2209407817 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb00011890 con 0x7fdb04123070 2020-03-19T23:11:07.915 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/2209407817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 msgr2=0x7fdb04127740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb0a0c6700 1 --2- 172.21.15.38:0/2209407817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb04127740 secure :-1 s=READY pgs=123 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).stop 2020-03-19T23:11:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/2209407817 shutdown_connections 2020-03-19T23:11:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb0a0c6700 1 --2- 172.21.15.38:0/2209407817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb04127740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/2209407817 >> 172.21.15.38:0/2209407817 conn(0x7fdb040950e0 msgr2=0x7fdb04091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:07.916 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/2209407817 shutdown_connections 2020-03-19T23:11:07.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.911+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/2209407817 wait complete. 2020-03-19T23:11:07.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb0a0c6700 1 Processor -- start 2020-03-19T23:11:07.924 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb0a0c6700 1 -- start start 2020-03-19T23:11:07.927 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb0a0c6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb0411f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb0a0c6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0410c0e0 con 0x7fdb0408e5c0 2020-03-19T23:11:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb0a0c6700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb04066100 con 0x7fdb04123070 2020-03-19T23:11:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb090c4700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdb0408e5c0 0x7fdb0411ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43942/0 (socket says 172.21.15.38:43942) 2020-03-19T23:11:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb090c4700 1 -- 172.21.15.38:0/3964331055 learned_addr learned my addr 172.21.15.38:0/3964331055 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.912+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb0411f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 -- 172.21.15.38:0/3964331055 >> v1:172.21.15.38:6789/0 conn(0x7fdb0408e5c0 legacy=0x7fdb0411ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:07.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 -- 172.21.15.38:0/3964331055 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0410be80 con 0x7fdb04123070 2020-03-19T23:11:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515541899 0 0) 0x7fdb0410c0e0 con 0x7fdb0408e5c0 2020-03-19T23:11:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> v2:172.21.15.38:3300/0 conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=READY pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb0000f660 con 0x7fdb04123070 2020-03-19T23:11:07.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0000f7c0 con 0x7fdb04123070 2020-03-19T23:11:07.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb04111770 con 0x7fdb04123070 2020-03-19T23:11:07.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fdb00007290 con 0x7fdb04123070 2020-03-19T23:11:07.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0410b1a0 con 0x7fdb04123070 2020-03-19T23:11:07.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.913+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fdb00007490 con 0x7fdb04123070 2020-03-19T23:11:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdaf67fc700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 0x7fdae402a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fdb00034f90 con 0x7fdb04123070 2020-03-19T23:11:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdb090c4700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 0x7fdae402a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdb090c4700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 0x7fdae402a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fdaf8001000 tx=0x7fdaf8001850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdb090c4700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 0x7fdae402a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fdaf8001000 tx=0x7fdaf8001850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb04056db0 con 0x7fdb04123070 2020-03-19T23:11:07.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.914+0000 7fdb090c4700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 0x7fdae402a6f0 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7fdaf8001000 tx=0x7fdaf8001850).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:07.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.918+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:07.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:07.918+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fdb00006870 con 0x7fdb04123070 2020-03-19T23:11:08.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.067+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb040912a0 con 0x7fdb04123070 2020-03-19T23:11:08.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.068+0000 7fdb088c3700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=THROTTLE_DONE pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.070 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.068+0000 7fdaf67fc700 1 -- 172.21.15.38:0/3964331055 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdb00006870 con 0x7fdb04123070 2020-03-19T23:11:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 msgr2=0x7fdae402a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 0x7fdae402a6f0 secure :-1 s=READY pgs=32 cs=0 l=1 rx=0x7fdaf8001000 tx=0x7fdaf8001850).stop 2020-03-19T23:11:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 msgr2=0x7fdb0411f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 secure :-1 s=READY pgs=125 cs=0 l=1 rx=0x7fdaf80010d0 tx=0x7fdb00007a40).stop 2020-03-19T23:11:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 shutdown_connections 2020-03-19T23:11:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fdae4028230 0x7fdae402a6f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 --2- 172.21.15.38:0/3964331055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb04123070 0x7fdb0411f150 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 >> 172.21.15.38:0/3964331055 conn(0x7fdb040950e0 msgr2=0x7fdb04060030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:08.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 shutdown_connections 2020-03-19T23:11:08.072 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.069+0000 7fdb0a0c6700 1 -- 172.21.15.38:0/3964331055 wait complete. 2020-03-19T23:11:08.072 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:08.080 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-03-19T23:11:08.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-03-19T23:11:08.080 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-03-19T23:11:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f32686cf700 1 Processor -- start 2020-03-19T23:11:08.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f32686cf700 1 -- start start 2020-03-19T23:11:08.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f32686cf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326008daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:08.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f32686cf700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32600605e0 con 0x7f326008d000 2020-03-19T23:11:08.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f32686cf700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326008ef70 con 0x7f32600907c0 2020-03-19T23:11:08.199 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f3265c6a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326008daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.199 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f3265c6a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326008daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60810/0 (socket says 172.21.15.38:60810) 2020-03-19T23:11:08.199 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f326646b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f326008d000 0x7f326008d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43948/0 (socket says 172.21.15.38:43948) 2020-03-19T23:11:08.199 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.192+0000 7f3265c6a700 1 -- 172.21.15.38:0/819520863 learned_addr learned my addr 172.21.15.38:0/819520863 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 -- 172.21.15.38:0/819520863 >> v1:172.21.15.38:6789/0 conn(0x7f326008d000 legacy=0x7f326008d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 -- 172.21.15.38:0/819520863 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260060810 con 0x7f32600907c0 2020-03-19T23:11:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265469700 1 -- 172.21.15.38:0/819520863 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503610872 0 0) 0x7f32600605e0 con 0x7f326008d000 2020-03-19T23:11:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 --2- 172.21.15.38:0/819520863 >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326008daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f3260090c70 tx=0x7f32540088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 --2- 172.21.15.38:0/819520863 >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326008daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f3260090c70 tx=0x7f32540088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 --2- 172.21.15.38:0/819520863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326008daf0 secure :-1 s=READY pgs=127 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f32540088e0).ready entity=mon.0 client_cookie=75676cd15c9669bb server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 --2- 172.21.15.38:0/819520863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326008daf0 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f32540088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265469700 1 -- 172.21.15.38:0/819520863 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f32540114b0 con 0x7f32600907c0 2020-03-19T23:11:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 --2- 172.21.15.38:0/819520863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326008daf0 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f32540088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265c6a700 1 --2- 172.21.15.38:0/819520863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326008daf0 secure :-1 s=THROTTLE_DONE pgs=127 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f32540088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265469700 1 -- 172.21.15.38:0/819520863 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254011610 con 0x7f32600907c0 2020-03-19T23:11:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f3265469700 1 -- 172.21.15.38:0/819520863 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f32540118b0 con 0x7f32600907c0 2020-03-19T23:11:08.201 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f32686cf700 1 -- 172.21.15.38:0/819520863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 msgr2=0x7f326008daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.193+0000 7f32686cf700 1 --2- 172.21.15.38:0/819520863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326008daf0 secure :-1 s=READY pgs=127 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f32540088e0).stop 2020-03-19T23:11:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.194+0000 7f32686cf700 1 -- 172.21.15.38:0/819520863 shutdown_connections 2020-03-19T23:11:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.194+0000 7f32686cf700 1 --2- 172.21.15.38:0/819520863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326008daf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.194+0000 7f32686cf700 1 -- 172.21.15.38:0/819520863 >> 172.21.15.38:0/819520863 conn(0x7f3260091540 msgr2=0x7f3260091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.194+0000 7f32686cf700 1 -- 172.21.15.38:0/819520863 shutdown_connections 2020-03-19T23:11:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.194+0000 7f32686cf700 1 -- 172.21.15.38:0/819520863 wait complete. 2020-03-19T23:11:08.202 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.194+0000 7f32686cf700 1 Processor -- start 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.194+0000 7f32686cf700 1 -- start start 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f32686cf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326011a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f32686cf700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3260123820 con 0x7f326008d000 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f32686cf700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3260064a50 con 0x7f32600907c0 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f326646b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f326008d000 0x7f3260065110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43952/0 (socket says 172.21.15.38:43952) 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f326646b700 1 -- 172.21.15.38:0/1953775247 learned_addr learned my addr 172.21.15.38:0/1953775247 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326011a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 -- 172.21.15.38:0/1953775247 >> v1:172.21.15.38:6789/0 conn(0x7f326008d000 legacy=0x7f3260065110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668352962 0 0) 0x7f3260123820 con 0x7f326008d000 2020-03-19T23:11:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 -- 172.21.15.38:0/1953775247 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326011aa30 con 0x7f32600907c0 2020-03-19T23:11:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> v2:172.21.15.38:3300/0 conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f325400f800 con 0x7f32600907c0 2020-03-19T23:11:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3260066220 con 0x7f32600907c0 2020-03-19T23:11:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3254007150 con 0x7f32600907c0 2020-03-19T23:11:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f32540073f0 con 0x7f32600907c0 2020-03-19T23:11:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.195+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326011a310 con 0x7f32600907c0 2020-03-19T23:11:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.196+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.196+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f32540056c0 con 0x7f32600907c0 2020-03-19T23:11:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.196+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.196+0000 7f325f7fe700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 0x7f324002a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.196+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f3254034890 con 0x7f32600907c0 2020-03-19T23:11:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.196+0000 7f326646b700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 0x7f324002a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.197+0000 7f326646b700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 0x7f324002a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f3258001170 tx=0x7f3258001770).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.197+0000 7f326646b700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 0x7f324002a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f3258001170 tx=0x7f3258001770).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.197+0000 7f326646b700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 0x7f324002a6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f3258001170 tx=0x7f3258001770).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.197+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326011b0e0 con 0x7f32600907c0 2020-03-19T23:11:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.200+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.200+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f32540075a0 con 0x7f32600907c0 2020-03-19T23:11:08.390 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.389+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3260056db0 con 0x7f32600907c0 2020-03-19T23:11:08.391 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.390+0000 7f3265c6a700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=THROTTLE_DONE pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.390+0000 7f325f7fe700 1 -- 172.21.15.38:0/1953775247 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+59995 (secure 0 0 0) 0x7f3254009de0 con 0x7f32600907c0 2020-03-19T23:11:08.392 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:08.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.391+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 msgr2=0x7f324002a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.391+0000 7f32686cf700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 0x7f324002a6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rx=0x7f3258001170 tx=0x7f3258001770).stop 2020-03-19T23:11:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.391+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 msgr2=0x7f326011a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.391+0000 7f32686cf700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rx=0x7f3260090c70 tx=0x7f3260067850).stop 2020-03-19T23:11:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.392+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 shutdown_connections 2020-03-19T23:11:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.392+0000 7f32686cf700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f3240028200 0x7f324002a6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.392+0000 7f32686cf700 1 --2- 172.21.15.38:0/1953775247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32600907c0 0x7f326011a5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.392+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 >> 172.21.15.38:0/1953775247 conn(0x7f3260091540 msgr2=0x7f3260125db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.392+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 shutdown_connections 2020-03-19T23:11:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.392+0000 7f32686cf700 1 -- 172.21.15.38:0/1953775247 wait complete. 2020-03-19T23:11:08.403 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":4,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":20200},{"type":"v1","addr":"172.21.15.38:6825","nonce":20200}]},"active_addr":"172.21.15.38:6825/20200","active_change":"2020-03-19T23:10:53.610259+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":1796046003}]},{"addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":238250007}]}]}} 2020-03-19T23:11:08.407 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-03-19T23:11:08.407 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-03-19T23:11:08.407 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.521+0000 7fde0092c700 1 Processor -- start 2020-03-19T23:11:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.521+0000 7fde0092c700 1 -- start start 2020-03-19T23:11:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.521+0000 7fde0092c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fddfc125ba0 0x7fddfc127f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.521+0000 7fde0092c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc090af0 con 0x7fddfc065d30 2020-03-19T23:11:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.521+0000 7fde0092c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddfc066820 con 0x7fddfc125ba0 2020-03-19T23:11:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.521+0000 7fddf9d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fddfc125ba0 0x7fddfc127f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fddfc125ba0 0x7fddfc127f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60820/0 (socket says 172.21.15.38:60820) 2020-03-19T23:11:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 -- 172.21.15.38:0/3243744403 learned_addr learned my addr 172.21.15.38:0/3243744403 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddfa59c700 1 --1- 172.21.15.38:0/3243744403 >> v1:172.21.15.38:6789/0 conn(0x7fddfc065d30 0x7fddfc066110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43958/0 (socket says 172.21.15.38:43958) 2020-03-19T23:11:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 -- 172.21.15.38:0/3243744403 >> v1:172.21.15.38:6789/0 conn(0x7fddfc065d30 legacy=0x7fddfc066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 -- 172.21.15.38:0/3243744403 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc08eec0 con 0x7fddfc125ba0 2020-03-19T23:11:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/3243744403 >> v2:172.21.15.38:3300/0 conn(0x7fddfc125ba0 0x7fddfc127f90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fddec002f30 tx=0x7fddec007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/3243744403 >> v2:172.21.15.38:3300/0 conn(0x7fddfc125ba0 0x7fddfc127f90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fddec002f30 tx=0x7fddec007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/3243744403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc125ba0 0x7fddfc127f90 secure :-1 s=READY pgs=131 cs=0 l=1 rx=0x7fddec002f30 tx=0x7fddec007a40).ready entity=mon.0 client_cookie=885d0bd307f4bb69 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/3243744403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc125ba0 0x7fddfc127f90 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7fddec002f30 tx=0x7fddec007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf959a700 1 -- 172.21.15.38:0/3243744403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fddec0114b0 con 0x7fddfc125ba0 2020-03-19T23:11:08.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/3243744403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc125ba0 0x7fddfc127f90 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7fddec002f30 tx=0x7fddec007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf959a700 1 -- 172.21.15.38:0/3243744403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec011610 con 0x7fddfc125ba0 2020-03-19T23:11:08.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/3243744403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc125ba0 0x7fddfc127f90 secure :-1 s=THROTTLE_DONE pgs=131 cs=0 l=1 rx=0x7fddec002f30 tx=0x7fddec007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fddf959a700 1 -- 172.21.15.38:0/3243744403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fddec011970 con 0x7fddfc125ba0 2020-03-19T23:11:08.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fde0092c700 1 -- 172.21.15.38:0/3243744403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc125ba0 msgr2=0x7fddfc127f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.522+0000 7fde0092c700 1 --2- 172.21.15.38:0/3243744403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc125ba0 0x7fddfc127f90 secure :-1 s=READY pgs=131 cs=0 l=1 rx=0x7fddec002f30 tx=0x7fddec007a40).stop 2020-03-19T23:11:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.523+0000 7fde0092c700 1 -- 172.21.15.38:0/3243744403 shutdown_connections 2020-03-19T23:11:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.523+0000 7fde0092c700 1 --2- 172.21.15.38:0/3243744403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc125ba0 0x7fddfc127f90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.523+0000 7fde0092c700 1 -- 172.21.15.38:0/3243744403 >> 172.21.15.38:0/3243744403 conn(0x7fddfc091540 msgr2=0x7fddfc091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.523+0000 7fde0092c700 1 -- 172.21.15.38:0/3243744403 shutdown_connections 2020-03-19T23:11:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.523+0000 7fde0092c700 1 -- 172.21.15.38:0/3243744403 wait complete. 2020-03-19T23:11:08.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fde0092c700 1 Processor -- start 2020-03-19T23:11:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fde0092c700 1 -- start start 2020-03-19T23:11:08.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fde0092c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fddfc065d30 0x7fddfc063ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:08.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fde0092c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc063990 con 0x7fddfc125ba0 2020-03-19T23:11:08.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fde0092c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddfc063bc0 con 0x7fddfc065d30 2020-03-19T23:11:08.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fddfa59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fddfc065d30 0x7fddfc063ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.533 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fddf9d9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fddfc125ba0 0x7fddfc064330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43964/0 (socket says 172.21.15.38:43964) 2020-03-19T23:11:08.533 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fddfa59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fddfc065d30 0x7fddfc063ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60822/0 (socket says 172.21.15.38:60822) 2020-03-19T23:11:08.533 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.524+0000 7fddf9d9b700 1 -- 172.21.15.38:0/1508195228 learned_addr learned my addr 172.21.15.38:0/1508195228 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:08.533 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 -- 172.21.15.38:0/1508195228 >> v1:172.21.15.38:6789/0 conn(0x7fddfc125ba0 legacy=0x7fddfc064330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.533 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 -- 172.21.15.38:0/1508195228 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc11b600 con 0x7fddfc065d30 2020-03-19T23:11:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225514919 0 0) 0x7fddfc063990 con 0x7fddfc125ba0 2020-03-19T23:11:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> v2:172.21.15.38:3300/0 conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> v2:172.21.15.38:3300/0 conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fddf0011430 con 0x7fddfc065d30 2020-03-19T23:11:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf0011590 con 0x7fddfc065d30 2020-03-19T23:11:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fddf0011830 con 0x7fddfc065d30 2020-03-19T23:11:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc065000 con 0x7fddfc065d30 2020-03-19T23:11:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc119b30 con 0x7fddfc065d30 2020-03-19T23:11:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.525+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.535 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fddf000b0a0 con 0x7fddfc065d30 2020-03-19T23:11:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddeb7fe700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 0x7fddd802a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fddf0033eb0 con 0x7fddfc065d30 2020-03-19T23:11:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 0x7fddd802a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 0x7fddd802a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fddec010280 tx=0x7fddec007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 0x7fddd802a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fddec010280 tx=0x7fddec007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fddf9d9b700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 0x7fddd802a710 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7fddec010280 tx=0x7fddec007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.526+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc11a5d0 con 0x7fddfc065d30 2020-03-19T23:11:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.530+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.530+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fddf0005e40 con 0x7fddfc065d30 2020-03-19T23:11:08.679 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.678+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddfc11a7d0 con 0x7fddfc065d30 2020-03-19T23:11:08.680 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.679+0000 7fddfa59c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=THROTTLE_DONE pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.680 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.679+0000 7fddeb7fe700 1 -- 172.21.15.38:0/1508195228 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8801 (secure 0 0 0) 0x7fddf0005e40 con 0x7fddfc065d30 2020-03-19T23:11:08.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:08.680 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:05.639978+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:08.682 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 msgr2=0x7fddd802a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.682 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 0x7fddd802a710 secure :-1 s=READY pgs=34 cs=0 l=1 rx=0x7fddec010280 tx=0x7fddec007a40).stop 2020-03-19T23:11:08.682 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 msgr2=0x7fddfc063ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.682 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rx=0x7fddfc066d90 tx=0x7fddf00108a0).stop 2020-03-19T23:11:08.682 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 shutdown_connections 2020-03-19T23:11:08.682 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddfc065d30 0x7fddfc063ed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.683 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 --2- 172.21.15.38:0/1508195228 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fddd8028250 0x7fddd802a710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.683 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 >> 172.21.15.38:0/1508195228 conn(0x7fddfc091540 msgr2=0x7fddfc125f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:08.683 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 shutdown_connections 2020-03-19T23:11:08.683 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.680+0000 7fde0092c700 1 -- 172.21.15.38:0/1508195228 wait complete. 2020-03-19T23:11:08.692 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-03-19T23:11:08.692 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:08.807 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.806+0000 7f67a6fa0700 1 Processor -- start 2020-03-19T23:11:08.807 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.806+0000 7f67a6fa0700 1 -- start start 2020-03-19T23:11:08.812 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a6fa0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f67a008c270 0x7f67a008c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:08.812 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a6fa0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a008d710 con 0x7f67a008cab0 2020-03-19T23:11:08.812 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a6fa0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a008cf40 con 0x7f67a008c270 2020-03-19T23:11:08.812 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f67a008c270 0x7f67a008c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.813 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f67a008c270 0x7f67a008c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60828/0 (socket says 172.21.15.38:60828) 2020-03-19T23:11:08.813 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 -- 172.21.15.38:0/4275364769 learned_addr learned my addr 172.21.15.38:0/4275364769 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:08.813 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a579d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f67a008cab0 0x7f67a0121e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43970/0 (socket says 172.21.15.38:43970) 2020-03-19T23:11:08.813 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 -- 172.21.15.38:0/4275364769 >> v1:172.21.15.38:6789/0 conn(0x7f67a008cab0 legacy=0x7f67a0121e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.813 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a4f9c700 1 -- 172.21.15.38:0/4275364769 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035336303 0 0) 0x7f67a008d710 con 0x7f67a008cab0 2020-03-19T23:11:08.813 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 -- 172.21.15.38:0/4275364769 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a008e010 con 0x7f67a008c270 2020-03-19T23:11:08.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/4275364769 >> v2:172.21.15.38:3300/0 conn(0x7f67a008c270 0x7f67a008c650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6790002f30 tx=0x7f6790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/4275364769 >> v2:172.21.15.38:3300/0 conn(0x7f67a008c270 0x7f67a008c650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6790002f30 tx=0x7f6790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.807+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/4275364769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008c270 0x7f67a008c650 secure :-1 s=READY pgs=135 cs=0 l=1 rx=0x7f6790002f30 tx=0x7f6790007a40).ready entity=mon.0 client_cookie=bc225842f51a9a89 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/4275364769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008c270 0x7f67a008c650 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7f6790002f30 tx=0x7f6790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a4f9c700 1 -- 172.21.15.38:0/4275364769 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6790013c20 con 0x7f67a008c270 2020-03-19T23:11:08.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/4275364769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008c270 0x7f67a008c650 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7f6790002f30 tx=0x7f6790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a4f9c700 1 -- 172.21.15.38:0/4275364769 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790013d80 con 0x7f67a008c270 2020-03-19T23:11:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/4275364769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008c270 0x7f67a008c650 secure :-1 s=THROTTLE_DONE pgs=135 cs=0 l=1 rx=0x7f6790002f30 tx=0x7f6790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a6fa0700 1 -- 172.21.15.38:0/4275364769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008c270 msgr2=0x7f67a008c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a4f9c700 1 -- 172.21.15.38:0/4275364769 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f679000c290 con 0x7f67a008c270 2020-03-19T23:11:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a6fa0700 1 --2- 172.21.15.38:0/4275364769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008c270 0x7f67a008c650 secure :-1 s=READY pgs=135 cs=0 l=1 rx=0x7f6790002f30 tx=0x7f6790007a40).stop 2020-03-19T23:11:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a6fa0700 1 -- 172.21.15.38:0/4275364769 shutdown_connections 2020-03-19T23:11:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a6fa0700 1 --2- 172.21.15.38:0/4275364769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008c270 0x7f67a008c650 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a6fa0700 1 -- 172.21.15.38:0/4275364769 >> 172.21.15.38:0/4275364769 conn(0x7f67a00953a0 msgr2=0x7f67a00914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a6fa0700 1 -- 172.21.15.38:0/4275364769 shutdown_connections 2020-03-19T23:11:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.808+0000 7f67a6fa0700 1 -- 172.21.15.38:0/4275364769 wait complete. 2020-03-19T23:11:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a6fa0700 1 Processor -- start 2020-03-19T23:11:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a6fa0700 1 -- start start 2020-03-19T23:11:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a6fa0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f67a008cab0 0x7f67a011f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a6fa0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a0060f70 con 0x7f67a008c270 2020-03-19T23:11:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a6fa0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a011ec60 con 0x7f67a008cab0 2020-03-19T23:11:08.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a5f9e700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f67a008c270 0x7f67a011e550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43972/0 (socket says 172.21.15.38:43972) 2020-03-19T23:11:08.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a5f9e700 1 -- 172.21.15.38:0/1630192461 learned_addr learned my addr 172.21.15.38:0/1630192461 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:08.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.809+0000 7f67a579d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f67a008cab0 0x7f67a011f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 -- 172.21.15.38:0/1630192461 >> v1:172.21.15.38:6789/0 conn(0x7f67a008c270 legacy=0x7f67a011e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 -- 172.21.15.38:0/1630192461 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a0061e10 con 0x7f67a008cab0 2020-03-19T23:11:08.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473395685 0 0) 0x7f67a0060f70 con 0x7f67a008c270 2020-03-19T23:11:08.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> v2:172.21.15.38:3300/0 conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.818 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> v2:172.21.15.38:3300/0 conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.818 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=READY pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.818 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.818 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f679c0114e0 con 0x7f67a008cab0 2020-03-19T23:11:08.818 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.818 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.819 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c011680 con 0x7f67a008cab0 2020-03-19T23:11:08.819 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f679c0119a0 con 0x7f67a008cab0 2020-03-19T23:11:08.819 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a011ee50 con 0x7f67a008cab0 2020-03-19T23:11:08.819 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a011f5c0 con 0x7f67a008cab0 2020-03-19T23:11:08.819 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.810+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.819 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f679c00b0a0 con 0x7f67a008cab0 2020-03-19T23:11:08.819 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f678effd700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 0x7f678002a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f679c034070 con 0x7f67a008cab0 2020-03-19T23:11:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 0x7f678002a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 0x7f678002a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6790012490 tx=0x7f6790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a0061c10 con 0x7f67a008cab0 2020-03-19T23:11:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 0x7f678002a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6790012490 tx=0x7f6790007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.811+0000 7f67a5f9e700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 0x7f678002a710 secure :-1 s=READY pgs=35 cs=0 l=1 rx=0x7f6790012490 tx=0x7f6790007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:08.821 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.815+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.821 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.815+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f679c011b00 con 0x7f67a008cab0 2020-03-19T23:11:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.962+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67a0056db0 con 0x7f67a008cab0 2020-03-19T23:11:08.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.963+0000 7f67a579d700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=THROTTLE_DONE pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:08.964 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.963+0000 7f678effd700 1 -- 172.21.15.38:0/1630192461 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8801 (secure 0 0 0) 0x7f679c007160 con 0x7f67a008cab0 2020-03-19T23:11:08.964 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:08.964 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:05.639978+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:08.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 msgr2=0x7f678002a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 0x7f678002a710 secure :-1 s=READY pgs=35 cs=0 l=1 rx=0x7f6790012490 tx=0x7f6790007a40).stop 2020-03-19T23:11:08.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 msgr2=0x7f67a011f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:08.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 secure :-1 s=READY pgs=137 cs=0 l=1 rx=0x7f67a00623f0 tx=0x7f679c0108f0).stop 2020-03-19T23:11:08.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 shutdown_connections 2020-03-19T23:11:08.966 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6780028250 0x7f678002a710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 --2- 172.21.15.38:0/1630192461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67a008cab0 0x7f67a011f160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:08.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 >> 172.21.15.38:0/1630192461 conn(0x7f67a00953a0 msgr2=0x7f67a01227f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:08.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 shutdown_connections 2020-03-19T23:11:08.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:08.964+0000 7f67a6fa0700 1 -- 172.21.15.38:0/1630192461 wait complete. 2020-03-19T23:11:08.976 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-03-19T23:11:09.090 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9db15d700 1 Processor -- start 2020-03-19T23:11:09.090 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9db15d700 1 -- start start 2020-03-19T23:11:09.095 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9db15d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:09.095 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9db15d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d4090e50 con 0x7ff9d411fcb0 2020-03-19T23:11:09.095 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9db15d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d4091190 con 0x7ff9d4061310 2020-03-19T23:11:09.095 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9da15b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.095 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9d995a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff9d411fcb0 0x7ff9d4060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43980/0 (socket says 172.21.15.38:43980) 2020-03-19T23:11:09.096 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.089+0000 7ff9d995a700 1 -- 172.21.15.38:0/2314258734 learned_addr learned my addr 172.21.15.38:0/2314258734 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:09.096 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 -- 172.21.15.38:0/2314258734 >> v1:172.21.15.38:6789/0 conn(0x7ff9d411fcb0 legacy=0x7ff9d4060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.096 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9d9159700 1 -- 172.21.15.38:0/2314258734 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147394727 0 0) 0x7ff9d4090e50 con 0x7ff9d411fcb0 2020-03-19T23:11:09.096 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 -- 172.21.15.38:0/2314258734 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d411f540 con 0x7ff9d4061310 2020-03-19T23:11:09.096 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2314258734 >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411f850 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ff9d40612d0 tx=0x7ff9c80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.096 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2314258734 >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411f850 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ff9d40612d0 tx=0x7ff9c80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.097 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2314258734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411f850 secure :-1 s=READY pgs=139 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9c80088e0).ready entity=mon.0 client_cookie=ce61da4abb009d38 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.097 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2314258734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411f850 secure :-1 s=THROTTLE_DONE pgs=139 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9c80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.097 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9d9159700 1 -- 172.21.15.38:0/2314258734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff9c80113f0 con 0x7ff9d4061310 2020-03-19T23:11:09.097 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2314258734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411f850 secure :-1 s=THROTTLE_DONE pgs=139 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9c80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.097 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2314258734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411f850 secure :-1 s=THROTTLE_DONE pgs=139 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9c80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.097 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9d9159700 1 -- 172.21.15.38:0/2314258734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8011550 con 0x7ff9d4061310 2020-03-19T23:11:09.097 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9d9159700 1 -- 172.21.15.38:0/2314258734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff9c8011830 con 0x7ff9d4061310 2020-03-19T23:11:09.098 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9db15d700 1 -- 172.21.15.38:0/2314258734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 msgr2=0x7ff9d411f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.098 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2314258734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411f850 secure :-1 s=READY pgs=139 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9c80088e0).stop 2020-03-19T23:11:09.098 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9db15d700 1 -- 172.21.15.38:0/2314258734 shutdown_connections 2020-03-19T23:11:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2314258734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411f850 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.090+0000 7ff9db15d700 1 -- 172.21.15.38:0/2314258734 >> 172.21.15.38:0/2314258734 conn(0x7ff9d4091390 msgr2=0x7ff9d4091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.091+0000 7ff9db15d700 1 -- 172.21.15.38:0/2314258734 shutdown_connections 2020-03-19T23:11:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.091+0000 7ff9db15d700 1 -- 172.21.15.38:0/2314258734 wait complete. 2020-03-19T23:11:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.091+0000 7ff9db15d700 1 Processor -- start 2020-03-19T23:11:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.091+0000 7ff9db15d700 1 -- start start 2020-03-19T23:11:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9db15d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.100 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9db15d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d4066900 con 0x7ff9d411fcb0 2020-03-19T23:11:09.100 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9db15d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d408f610 con 0x7ff9d4061310 2020-03-19T23:11:09.100 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.100 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9d995a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff9d411fcb0 0x7ff9d411bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43984/0 (socket says 172.21.15.38:43984) 2020-03-19T23:11:09.100 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9d995a700 1 -- 172.21.15.38:0/2747653430 learned_addr learned my addr 172.21.15.38:0/2747653430 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:09.100 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 -- 172.21.15.38:0/2747653430 >> v1:172.21.15.38:6789/0 conn(0x7ff9d411fcb0 legacy=0x7ff9d411bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.101 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 -- 172.21.15.38:0/2747653430 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d410c590 con 0x7ff9d4061310 2020-03-19T23:11:09.101 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590491195 0 0) 0x7ff9d4066900 con 0x7ff9d411fcb0 2020-03-19T23:11:09.101 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.101 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> v2:172.21.15.38:3300/0 conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.101 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=READY pgs=141 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.101 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.102 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff9c800f740 con 0x7ff9d4061310 2020-03-19T23:11:09.102 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.102 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.102 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8007150 con 0x7ff9d4061310 2020-03-19T23:11:09.102 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d4066c60 con 0x7ff9d4061310 2020-03-19T23:11:09.102 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff9c80073f0 con 0x7ff9d4061310 2020-03-19T23:11:09.103 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.092+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d40660e0 con 0x7ff9d4061310 2020-03-19T23:11:09.103 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.103 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7ff9c80056c0 con 0x7ff9d4061310 2020-03-19T23:11:09.103 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9da15b700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=THROTTLE_DONE pgs=141 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.103 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9c77fe700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 0x7ff9b402a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.103 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7ff9c80348e0 con 0x7ff9d4061310 2020-03-19T23:11:09.103 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9d995a700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 0x7ff9b402a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.104 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7ff9d4055b30 0x7ff9d4055f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.104 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 --> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9d4056db0 con 0x7ff9d4055b30 2020-03-19T23:11:09.104 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.093+0000 7ff9d995a700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 0x7ff9b402a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff9d0005880 tx=0x7ff9d0001a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.104 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.094+0000 7ff9d995a700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 0x7ff9b402a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff9d0005880 tx=0x7ff9d0001a80).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.104 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.094+0000 7ff9d995a700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 0x7ff9b402a6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rx=0x7ff9d0005880 tx=0x7ff9d0001a80).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.104 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.094+0000 7ff9da95c700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7ff9d4055b30 0x7ff9d4055f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.105 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.094+0000 7ff9da95c700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7ff9d4055b30 0x7ff9d4055f10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.105 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.095+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== osd.0 v2:172.21.15.38:6808/20729 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7ff9d4056db0 con 0x7ff9d4055b30 2020-03-19T23:11:09.114 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.113+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 --> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9d4123450 con 0x7ff9d4055b30 2020-03-19T23:11:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.113+0000 7ff9c77fe700 1 -- 172.21.15.38:0/2747653430 <== osd.0 v2:172.21.15.38:6808/20729 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9d4123450 con 0x7ff9d4055b30 2020-03-19T23:11:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.113+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7ff9d4055b30 msgr2=0x7ff9d4055f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.113+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7ff9d4055b30 0x7ff9d4055f10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 msgr2=0x7ff9b402a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 0x7ff9b402a6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rx=0x7ff9d0005880 tx=0x7ff9d0001a80).stop 2020-03-19T23:11:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 msgr2=0x7ff9d411b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 secure :-1 s=READY pgs=141 cs=0 l=1 rx=0x7ff9d40612d0 tx=0x7ff9d411f6d0).stop 2020-03-19T23:11:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 shutdown_connections 2020-03-19T23:11:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7ff9d4055b30 0x7ff9d4055f10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.116 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d4061310 0x7ff9d411b880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 --2- 172.21.15.38:0/2747653430 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff9b4028200 0x7ff9b402a6c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 >> 172.21.15.38:0/2747653430 conn(0x7ff9d4091390 msgr2=0x7ff9d4060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 shutdown_connections 2020-03-19T23:11:09.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.114+0000 7ff9db15d700 1 -- 172.21.15.38:0/2747653430 wait complete. 2020-03-19T23:11:09.124 INFO:teuthology.orchestra.run.smithi038.stdout:34359738372 2020-03-19T23:11:09.125 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-03-19T23:11:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.236+0000 7fc78afbe700 1 Processor -- start 2020-03-19T23:11:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.236+0000 7fc78afbe700 1 -- start start 2020-03-19T23:11:09.244 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc78afbe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc78411f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:09.244 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc78afbe700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7840911c0 con 0x7fc78411fab0 2020-03-19T23:11:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc78afbe700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc784067130 con 0x7fc784091eb0 2020-03-19T23:11:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc78411f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc78411f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60850/0 (socket says 172.21.15.38:60850) 2020-03-19T23:11:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 -- 172.21.15.38:0/2982306399 learned_addr learned my addr 172.21.15.38:0/2982306399 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc783fff700 1 --1- 172.21.15.38:0/2982306399 >> v1:172.21.15.38:6789/0 conn(0x7fc78411fab0 0x7fc784060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43992/0 (socket says 172.21.15.38:43992) 2020-03-19T23:11:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 -- 172.21.15.38:0/2982306399 >> v1:172.21.15.38:6789/0 conn(0x7fc78411fab0 legacy=0x7fc784060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 -- 172.21.15.38:0/2982306399 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78411f340 con 0x7fc784091eb0 2020-03-19T23:11:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc7837fe700 1 -- 172.21.15.38:0/2982306399 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625315309 0 0) 0x7fc7840911c0 con 0x7fc78411fab0 2020-03-19T23:11:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2982306399 >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc78411f650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc774002f30 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2982306399 >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc78411f650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc774002f30 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2982306399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc78411f650 secure :-1 s=READY pgs=143 cs=0 l=1 rx=0x7fc774002f30 tx=0x7fc774007a40).ready entity=mon.0 client_cookie=e178fc38160d499a server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2982306399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc78411f650 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7fc774002f30 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc7837fe700 1 -- 172.21.15.38:0/2982306399 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc7740114b0 con 0x7fc784091eb0 2020-03-19T23:11:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2982306399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc78411f650 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7fc774002f30 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2982306399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc78411f650 secure :-1 s=THROTTLE_DONE pgs=143 cs=0 l=1 rx=0x7fc774002f30 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc7837fe700 1 -- 172.21.15.38:0/2982306399 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774011610 con 0x7fc784091eb0 2020-03-19T23:11:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc7837fe700 1 -- 172.21.15.38:0/2982306399 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc7740118f0 con 0x7fc784091eb0 2020-03-19T23:11:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc78afbe700 1 -- 172.21.15.38:0/2982306399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 msgr2=0x7fc78411f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.237+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2982306399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc78411f650 secure :-1 s=READY pgs=143 cs=0 l=1 rx=0x7fc774002f30 tx=0x7fc774007a40).stop 2020-03-19T23:11:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.238+0000 7fc78afbe700 1 -- 172.21.15.38:0/2982306399 shutdown_connections 2020-03-19T23:11:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.238+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2982306399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc78411f650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.238+0000 7fc78afbe700 1 -- 172.21.15.38:0/2982306399 >> 172.21.15.38:0/2982306399 conn(0x7fc784091390 msgr2=0x7fc7840917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.238+0000 7fc78afbe700 1 -- 172.21.15.38:0/2982306399 shutdown_connections 2020-03-19T23:11:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.238+0000 7fc78afbe700 1 -- 172.21.15.38:0/2982306399 wait complete. 2020-03-19T23:11:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc78afbe700 1 Processor -- start 2020-03-19T23:11:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc78afbe700 1 -- start start 2020-03-19T23:11:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc78afbe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc784064dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc78afbe700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7840664f0 con 0x7fc78411fab0 2020-03-19T23:11:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc78afbe700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78410bc90 con 0x7fc784091eb0 2020-03-19T23:11:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc788d5a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc784064dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc783fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc78411fab0 0x7fc784065220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43996/0 (socket says 172.21.15.38:43996) 2020-03-19T23:11:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.239+0000 7fc783fff700 1 -- 172.21.15.38:0/2649755074 learned_addr learned my addr 172.21.15.38:0/2649755074 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 -- 172.21.15.38:0/2649755074 >> v1:172.21.15.38:6789/0 conn(0x7fc78411fab0 legacy=0x7fc784065220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 -- 172.21.15.38:0/2649755074 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784066790 con 0x7fc784091eb0 2020-03-19T23:11:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> v2:172.21.15.38:3300/0 conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc77400f920 con 0x7fc784091eb0 2020-03-19T23:11:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774007150 con 0x7fc784091eb0 2020-03-19T23:11:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78411ce10 con 0x7fc784091eb0 2020-03-19T23:11:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc774007430 con 0x7fc784091eb0 2020-03-19T23:11:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7840661d0 con 0x7fc784091eb0 2020-03-19T23:11:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.240+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fc7740056c0 con 0x7fc784091eb0 2020-03-19T23:11:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc781ffb700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 0x7fc76402a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc784056db0 con 0x7fc784091eb0 2020-03-19T23:11:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fc7740348b0 con 0x7fc784091eb0 2020-03-19T23:11:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc783fff700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 0x7fc76402a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc788d5a700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=THROTTLE_DONE pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc781ffb700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7fc76402d830 0x7fc76402fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 --> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7640301e0 con 0x7fc76402d830 2020-03-19T23:11:09.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc774005950 con 0x7fc784091eb0 2020-03-19T23:11:09.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc783fff700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 0x7fc76402a780 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc78411f910 tx=0x7fc778001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc78955b700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7fc76402d830 0x7fc76402fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc783fff700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 0x7fc76402a780 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc78411f910 tx=0x7fc778001670).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.241+0000 7fc783fff700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 0x7fc76402a780 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7fc78411f910 tx=0x7fc778001670).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.242+0000 7fc78955b700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7fc76402d830 0x7fc76402fc50 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.242+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== osd.1 v2:172.21.15.38:6800/20730 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7fc7640301e0 con 0x7fc76402d830 2020-03-19T23:11:09.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 --> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7840693f0 con 0x7fc76402d830 2020-03-19T23:11:09.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc781ffb700 1 -- 172.21.15.38:0/2649755074 <== osd.1 v2:172.21.15.38:6800/20730 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc7840693f0 con 0x7fc76402d830 2020-03-19T23:11:09.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7fc76402d830 msgr2=0x7fc76402fc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7fc76402d830 0x7fc76402fc50 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 msgr2=0x7fc76402a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 0x7fc76402a780 secure :-1 s=READY pgs=37 cs=0 l=1 rx=0x7fc78411f910 tx=0x7fc778001670).stop 2020-03-19T23:11:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 msgr2=0x7fc784064dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.262+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rx=0x7fc78410b760 tx=0x7fc774007a40).stop 2020-03-19T23:11:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.263+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 shutdown_connections 2020-03-19T23:11:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.263+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7fc76402d830 0x7fc76402fc50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.263+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc784091eb0 0x7fc784064dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.263+0000 7fc78afbe700 1 --2- 172.21.15.38:0/2649755074 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7640282c0 0x7fc76402a780 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.263+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 >> 172.21.15.38:0/2649755074 conn(0x7fc784091390 msgr2=0x7fc784060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.263+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 shutdown_connections 2020-03-19T23:11:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.263+0000 7fc78afbe700 1 -- 172.21.15.38:0/2649755074 wait complete. 2020-03-19T23:11:09.274 INFO:teuthology.orchestra.run.smithi038.stdout:34359738372 2020-03-19T23:11:09.275 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-03-19T23:11:09.399 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f7325b80700 1 Processor -- start 2020-03-19T23:11:09.399 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f7325b80700 1 -- start start 2020-03-19T23:11:09.405 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f7325b80700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f732011fd60 0x7f7320127a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:09.405 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f7325b80700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73200911c0 con 0x7f7320091eb0 2020-03-19T23:11:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f7325b80700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7320067130 con 0x7f732011fd60 2020-03-19T23:11:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f731effd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f732011fd60 0x7f7320127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f731f7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7320091eb0 0x7f732011f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44002/0 (socket says 172.21.15.38:44002) 2020-03-19T23:11:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f731effd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f732011fd60 0x7f7320127a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60862/0 (socket says 172.21.15.38:60862) 2020-03-19T23:11:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.388+0000 7f731f7fe700 1 -- 172.21.15.38:0/3996260377 learned_addr learned my addr 172.21.15.38:0/3996260377 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 -- 172.21.15.38:0/3996260377 >> v1:172.21.15.38:6789/0 conn(0x7f7320091eb0 legacy=0x7f732011f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731e7fc700 1 -- 172.21.15.38:0/3996260377 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792383433 0 0) 0x7f73200911c0 con 0x7f7320091eb0 2020-03-19T23:11:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 -- 172.21.15.38:0/3996260377 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732011f340 con 0x7f732011fd60 2020-03-19T23:11:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 --2- 172.21.15.38:0/3996260377 >> v2:172.21.15.38:3300/0 conn(0x7f732011fd60 0x7f7320127a70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7314002f30 tx=0x7f7314007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 --2- 172.21.15.38:0/3996260377 >> v2:172.21.15.38:3300/0 conn(0x7f732011fd60 0x7f7320127a70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7314002f30 tx=0x7f7314007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 --2- 172.21.15.38:0/3996260377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732011fd60 0x7f7320127a70 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f7314002f30 tx=0x7f7314007a40).ready entity=mon.0 client_cookie=1995fcce25b91b17 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.407 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 --2- 172.21.15.38:0/3996260377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732011fd60 0x7f7320127a70 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f7314002f30 tx=0x7f7314007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.408 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731e7fc700 1 -- 172.21.15.38:0/3996260377 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7314008e80 con 0x7f732011fd60 2020-03-19T23:11:09.409 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 --2- 172.21.15.38:0/3996260377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732011fd60 0x7f7320127a70 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f7314002f30 tx=0x7f7314007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.409 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731e7fc700 1 -- 172.21.15.38:0/3996260377 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73140113f0 con 0x7f732011fd60 2020-03-19T23:11:09.409 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731effd700 1 --2- 172.21.15.38:0/3996260377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732011fd60 0x7f7320127a70 secure :-1 s=THROTTLE_DONE pgs=147 cs=0 l=1 rx=0x7f7314002f30 tx=0x7f7314007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.409 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f731e7fc700 1 -- 172.21.15.38:0/3996260377 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7314011690 con 0x7f732011fd60 2020-03-19T23:11:09.409 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f7325b80700 1 -- 172.21.15.38:0/3996260377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732011fd60 msgr2=0x7f7320127a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.409 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f7325b80700 1 --2- 172.21.15.38:0/3996260377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732011fd60 0x7f7320127a70 secure :-1 s=READY pgs=147 cs=0 l=1 rx=0x7f7314002f30 tx=0x7f7314007a40).stop 2020-03-19T23:11:09.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f7325b80700 1 -- 172.21.15.38:0/3996260377 shutdown_connections 2020-03-19T23:11:09.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f7325b80700 1 --2- 172.21.15.38:0/3996260377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732011fd60 0x7f7320127a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f7325b80700 1 -- 172.21.15.38:0/3996260377 >> 172.21.15.38:0/3996260377 conn(0x7f7320091390 msgr2=0x7f73200917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f7325b80700 1 -- 172.21.15.38:0/3996260377 shutdown_connections 2020-03-19T23:11:09.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.389+0000 7f7325b80700 1 -- 172.21.15.38:0/3996260377 wait complete. 2020-03-19T23:11:09.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.390+0000 7f7325b80700 1 Processor -- start 2020-03-19T23:11:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.390+0000 7f7325b80700 1 -- start start 2020-03-19T23:11:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f7325b80700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7320091eb0 0x7f7320066a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f7325b80700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7320062b20 con 0x7f732011fd60 2020-03-19T23:11:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f7325b80700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7320062810 con 0x7f7320091eb0 2020-03-19T23:11:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7320091eb0 0x7f7320066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731effd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f732011fd60 0x7f7320125430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44008/0 (socket says 172.21.15.38:44008) 2020-03-19T23:11:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7320091eb0 0x7f7320066a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60866/0 (socket says 172.21.15.38:60866) 2020-03-19T23:11:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731effd700 1 -- 172.21.15.38:0/3063753534 learned_addr learned my addr 172.21.15.38:0/3063753534 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 -- 172.21.15.38:0/3063753534 >> v1:172.21.15.38:6789/0 conn(0x7f732011fd60 legacy=0x7f7320125430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212498717 0 0) 0x7f7320062b20 con 0x7f732011fd60 2020-03-19T23:11:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 -- 172.21.15.38:0/3063753534 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73201265e0 con 0x7f7320091eb0 2020-03-19T23:11:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> v2:172.21.15.38:3300/0 conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> v2:172.21.15.38:3300/0 conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=READY pgs=149 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7310005b20 con 0x7f7320091eb0 2020-03-19T23:11:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7310005c80 con 0x7f7320091eb0 2020-03-19T23:11:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73201249b0 con 0x7f7320091eb0 2020-03-19T23:11:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f73100060c0 con 0x7f7320091eb0 2020-03-19T23:11:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.391+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7320125b40 con 0x7f7320091eb0 2020-03-19T23:11:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f731000a320 con 0x7f7320091eb0 2020-03-19T23:11:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731f7fe700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=THROTTLE_DONE pgs=149 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731cff9700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 0x7f730002a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f7310034000 con 0x7f7320091eb0 2020-03-19T23:11:09.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731effd700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 0x7f730002a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f7325b80700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7320055b30 0x7f7320055f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 --> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73201276d0 con 0x7f7320055b30 2020-03-19T23:11:09.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731effd700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 0x7f730002a770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7314002ed0 tx=0x7f7314007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731ffff700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7320055b30 0x7f7320055f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731effd700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 0x7f730002a770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7314002ed0 tx=0x7f7314007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.392+0000 7f731effd700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 0x7f730002a770 secure :-1 s=READY pgs=38 cs=0 l=1 rx=0x7f7314002ed0 tx=0x7f7314007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.393+0000 7f731ffff700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7320055b30 0x7f7320055f10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.394+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== osd.2 v2:172.21.15.38:6816/20739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7f73201276d0 con 0x7f7320055b30 2020-03-19T23:11:09.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.411+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 --> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7320056db0 con 0x7f7320055b30 2020-03-19T23:11:09.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f731cff9700 1 -- 172.21.15.38:0/3063753534 <== osd.2 v2:172.21.15.38:6816/20739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7320056db0 con 0x7f7320055b30 2020-03-19T23:11:09.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7320055b30 msgr2=0x7f7320055f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7320055b30 0x7f7320055f10 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 msgr2=0x7f730002a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 0x7f730002a770 secure :-1 s=READY pgs=38 cs=0 l=1 rx=0x7f7314002ed0 tx=0x7f7314007a40).stop 2020-03-19T23:11:09.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 msgr2=0x7f7320066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 secure :-1 s=READY pgs=149 cs=0 l=1 rx=0x7f7310000b80 tx=0x7f7310010930).stop 2020-03-19T23:11:09.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 shutdown_connections 2020-03-19T23:11:09.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7320091eb0 0x7f7320066a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7320055b30 0x7f7320055f10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 --2- 172.21.15.38:0/3063753534 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f73000282b0 0x7f730002a770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 >> 172.21.15.38:0/3063753534 conn(0x7f7320091390 msgr2=0x7f7320060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 shutdown_connections 2020-03-19T23:11:09.429 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.412+0000 7f7325b80700 1 -- 172.21.15.38:0/3063753534 wait complete. 2020-03-19T23:11:09.430 INFO:teuthology.orchestra.run.smithi038.stdout:34359738373 2020-03-19T23:11:09.432 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-03-19T23:11:09.546 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.544+0000 7f0d061ac700 1 Processor -- start 2020-03-19T23:11:09.546 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.544+0000 7f0d061ac700 1 -- start start 2020-03-19T23:11:09.553 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.544+0000 7f0d061ac700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:09.553 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.544+0000 7f0d061ac700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000911c0 con 0x7f0d0011fab0 2020-03-19T23:11:09.553 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.544+0000 7f0d061ac700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d00067130 con 0x7f0d00091eb0 2020-03-19T23:11:09.553 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.544+0000 7f0cff7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.553 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cf7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0d0011fab0 0x7f0d00060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44014/0 (socket says 172.21.15.38:44014) 2020-03-19T23:11:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60874/0 (socket says 172.21.15.38:60874) 2020-03-19T23:11:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cf7fff700 1 -- 172.21.15.38:0/3241896652 learned_addr learned my addr 172.21.15.38:0/3241896652 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 -- 172.21.15.38:0/3241896652 >> v1:172.21.15.38:6789/0 conn(0x7f0d0011fab0 legacy=0x7f0d00060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cfeffd700 1 -- 172.21.15.38:0/3241896652 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147330854 0 0) 0x7f0d000911c0 con 0x7f0d0011fab0 2020-03-19T23:11:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 -- 172.21.15.38:0/3241896652 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0011f340 con 0x7f0d00091eb0 2020-03-19T23:11:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/3241896652 >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011f650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/3241896652 >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011f650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/3241896652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011f650 secure :-1 s=READY pgs=151 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).ready entity=mon.0 client_cookie=b0a0c182d180bb3c server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/3241896652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011f650 secure :-1 s=THROTTLE_DONE pgs=151 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cfeffd700 1 -- 172.21.15.38:0/3241896652 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0cf0008e80 con 0x7f0d00091eb0 2020-03-19T23:11:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/3241896652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011f650 secure :-1 s=THROTTLE_DONE pgs=151 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.556 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cfeffd700 1 -- 172.21.15.38:0/3241896652 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0013600 con 0x7f0d00091eb0 2020-03-19T23:11:09.556 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/3241896652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011f650 secure :-1 s=THROTTLE_DONE pgs=151 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.556 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0cfeffd700 1 -- 172.21.15.38:0/3241896652 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0cf00138a0 con 0x7f0d00091eb0 2020-03-19T23:11:09.556 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0d061ac700 1 -- 172.21.15.38:0/3241896652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 msgr2=0x7f0d0011f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.557 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.545+0000 7f0d061ac700 1 --2- 172.21.15.38:0/3241896652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011f650 secure :-1 s=READY pgs=151 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).stop 2020-03-19T23:11:09.557 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.546+0000 7f0d061ac700 1 -- 172.21.15.38:0/3241896652 shutdown_connections 2020-03-19T23:11:09.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.546+0000 7f0d061ac700 1 --2- 172.21.15.38:0/3241896652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011f650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.546+0000 7f0d061ac700 1 -- 172.21.15.38:0/3241896652 >> 172.21.15.38:0/3241896652 conn(0x7f0d00091390 msgr2=0x7f0d000917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.546+0000 7f0d061ac700 1 -- 172.21.15.38:0/3241896652 shutdown_connections 2020-03-19T23:11:09.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.546+0000 7f0d061ac700 1 -- 172.21.15.38:0/3241896652 wait complete. 2020-03-19T23:11:09.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.546+0000 7f0d061ac700 1 Processor -- start 2020-03-19T23:11:09.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0d061ac700 1 -- start start 2020-03-19T23:11:09.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0d061ac700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0d061ac700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d00066a30 con 0x7f0d0011fab0 2020-03-19T23:11:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0d061ac700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d00066c90 con 0x7f0d00091eb0 2020-03-19T23:11:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cff7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cf7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0d0011fab0 0x7f0d0011bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44020/0 (socket says 172.21.15.38:44020) 2020-03-19T23:11:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cff7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60878/0 (socket says 172.21.15.38:60878) 2020-03-19T23:11:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cf7fff700 1 -- 172.21.15.38:0/1506258540 learned_addr learned my addr 172.21.15.38:0/1506258540 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cff7fe700 1 -- 172.21.15.38:0/1506258540 >> v1:172.21.15.38:6789/0 conn(0x7f0d0011fab0 legacy=0x7f0d0011bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186872298 0 0) 0x7f0d00066a30 con 0x7f0d0011fab0 2020-03-19T23:11:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cff7fe700 1 -- 172.21.15.38:0/1506258540 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0010b520 con 0x7f0d00091eb0 2020-03-19T23:11:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> v2:172.21.15.38:3300/0 conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.547+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=READY pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0cf0008ea0 con 0x7f0d00091eb0 2020-03-19T23:11:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0007150 con 0x7f0d00091eb0 2020-03-19T23:11:09.564 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d00064b40 con 0x7f0d00091eb0 2020-03-19T23:11:09.564 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0cf0007430 con 0x7f0d00091eb0 2020-03-19T23:11:09.564 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d00065460 con 0x7f0d00091eb0 2020-03-19T23:11:09.564 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.564 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f0cf00056c0 con 0x7f0d00091eb0 2020-03-19T23:11:09.564 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cfd7fa700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 0x7f0ce402a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f0cf0036a10 con 0x7f0d00091eb0 2020-03-19T23:11:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.548+0000 7f0cf7fff700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 0x7f0ce402a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.549+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0010b260 con 0x7f0d00091eb0 2020-03-19T23:11:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.549+0000 7f0cf7fff700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 0x7f0ce402a770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0ce8005500 tx=0x7f0ce80016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.549+0000 7f0cf7fff700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 0x7f0ce402a770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0ce8005500 tx=0x7f0ce80016f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.549+0000 7f0cf7fff700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 0x7f0ce402a770 secure :-1 s=READY pgs=39 cs=0 l=1 rx=0x7f0ce8005500 tx=0x7f0ce80016f0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.552+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.552+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f0cf000b970 con 0x7f0d00091eb0 2020-03-19T23:11:09.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.702+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0d00056db0 con 0x7f0d00091eb0 2020-03-19T23:11:09.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.703+0000 7f0cff7fe700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=THROTTLE_DONE pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:09.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.703+0000 7f0cfd7fa700 1 -- 172.21.15.38:0/1506258540 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0cf0009df0 con 0x7f0d00091eb0 2020-03-19T23:11:09.706 INFO:teuthology.orchestra.run.smithi038.stdout:34359738371 2020-03-19T23:11:09.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 msgr2=0x7f0ce402a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 0x7f0ce402a770 secure :-1 s=READY pgs=39 cs=0 l=1 rx=0x7f0ce8005500 tx=0x7f0ce80016f0).stop 2020-03-19T23:11:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 msgr2=0x7f0d0011b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 secure :-1 s=READY pgs=153 cs=0 l=1 rx=0x7f0cf0002f30 tx=0x7f0cf0007a40).stop 2020-03-19T23:11:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 shutdown_connections 2020-03-19T23:11:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d00091eb0 0x7f0d0011b820 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 --2- 172.21.15.38:0/1506258540 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0ce40282b0 0x7f0ce402a770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 >> 172.21.15.38:0/1506258540 conn(0x7f0d00091390 msgr2=0x7f0d000928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 shutdown_connections 2020-03-19T23:11:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:09.704+0000 7f0d061ac700 1 -- 172.21.15.38:0/1506258540 wait complete. 2020-03-19T23:11:09.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2020-03-19T23:11:10.718 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-03-19T23:11:10.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.843+0000 7fd5c48b5700 1 Processor -- start 2020-03-19T23:11:10.845 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.843+0000 7fd5c48b5700 1 -- start start 2020-03-19T23:11:10.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5c48b5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5c011fd60 0x7fd5c0127a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5c48b5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c00911c0 con 0x7fd5c0091eb0 2020-03-19T23:11:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5c48b5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c0067130 con 0x7fd5c011fd60 2020-03-19T23:11:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5c011fd60 0x7fd5c0127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5be59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd5c0091eb0 0x7fd5c011f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44024/0 (socket says 172.21.15.38:44024) 2020-03-19T23:11:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5c011fd60 0x7fd5c0127a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60886/0 (socket says 172.21.15.38:60886) 2020-03-19T23:11:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5be59c700 1 -- 172.21.15.38:0/1032350706 learned_addr learned my addr 172.21.15.38:0/1032350706 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 -- 172.21.15.38:0/1032350706 >> v1:172.21.15.38:6789/0 conn(0x7fd5c0091eb0 legacy=0x7fd5c011f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 -- 172.21.15.38:0/1032350706 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c011f340 con 0x7fd5c011fd60 2020-03-19T23:11:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bd59a700 1 -- 172.21.15.38:0/1032350706 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755853973 0 0) 0x7fd5c00911c0 con 0x7fd5c0091eb0 2020-03-19T23:11:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/1032350706 >> v2:172.21.15.38:3300/0 conn(0x7fd5c011fd60 0x7fd5c0127a70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fd5b0002f30 tx=0x7fd5b0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/1032350706 >> v2:172.21.15.38:3300/0 conn(0x7fd5c011fd60 0x7fd5c0127a70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fd5b0002f30 tx=0x7fd5b0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/1032350706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c011fd60 0x7fd5c0127a70 secure :-1 s=READY pgs=155 cs=0 l=1 rx=0x7fd5b0002f30 tx=0x7fd5b0007a40).ready entity=mon.0 client_cookie=7b0de8f6303d4139 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:10.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/1032350706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c011fd60 0x7fd5c0127a70 secure :-1 s=THROTTLE_DONE pgs=155 cs=0 l=1 rx=0x7fd5b0002f30 tx=0x7fd5b0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bd59a700 1 -- 172.21.15.38:0/1032350706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd5b0008e80 con 0x7fd5c011fd60 2020-03-19T23:11:10.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/1032350706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c011fd60 0x7fd5c0127a70 secure :-1 s=THROTTLE_DONE pgs=155 cs=0 l=1 rx=0x7fd5b0002f30 tx=0x7fd5b0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/1032350706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c011fd60 0x7fd5c0127a70 secure :-1 s=THROTTLE_DONE pgs=155 cs=0 l=1 rx=0x7fd5b0002f30 tx=0x7fd5b0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bd59a700 1 -- 172.21.15.38:0/1032350706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b00113f0 con 0x7fd5c011fd60 2020-03-19T23:11:10.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.844+0000 7fd5bd59a700 1 -- 172.21.15.38:0/1032350706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd5b0011690 con 0x7fd5c011fd60 2020-03-19T23:11:10.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.845+0000 7fd5c48b5700 1 -- 172.21.15.38:0/1032350706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c011fd60 msgr2=0x7fd5c0127a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:10.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.845+0000 7fd5c48b5700 1 --2- 172.21.15.38:0/1032350706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c011fd60 0x7fd5c0127a70 secure :-1 s=READY pgs=155 cs=0 l=1 rx=0x7fd5b0002f30 tx=0x7fd5b0007a40).stop 2020-03-19T23:11:10.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.845+0000 7fd5c48b5700 1 -- 172.21.15.38:0/1032350706 shutdown_connections 2020-03-19T23:11:10.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.845+0000 7fd5c48b5700 1 --2- 172.21.15.38:0/1032350706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c011fd60 0x7fd5c0127a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:10.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.845+0000 7fd5c48b5700 1 -- 172.21.15.38:0/1032350706 >> 172.21.15.38:0/1032350706 conn(0x7fd5c0091390 msgr2=0x7fd5c00917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:10.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.845+0000 7fd5c48b5700 1 -- 172.21.15.38:0/1032350706 shutdown_connections 2020-03-19T23:11:10.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.845+0000 7fd5c48b5700 1 -- 172.21.15.38:0/1032350706 wait complete. 2020-03-19T23:11:10.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5c48b5700 1 Processor -- start 2020-03-19T23:11:10.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5c48b5700 1 -- start start 2020-03-19T23:11:10.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5c48b5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5c0091eb0 0x7fd5c0064f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:10.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5c48b5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c010af70 con 0x7fd5c011fd60 2020-03-19T23:11:10.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5c48b5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c010bdd0 con 0x7fd5c0091eb0 2020-03-19T23:11:10.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5be59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5c0091eb0 0x7fd5c0064f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:10.857 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5bdd9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd5c011fd60 0x7fd5c00653c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44030/0 (socket says 172.21.15.38:44030) 2020-03-19T23:11:10.857 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5be59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5c0091eb0 0x7fd5c0064f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60888/0 (socket says 172.21.15.38:60888) 2020-03-19T23:11:10.857 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.846+0000 7fd5bdd9b700 1 -- 172.21.15.38:0/53584276 learned_addr learned my addr 172.21.15.38:0/53584276 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:10.857 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 -- 172.21.15.38:0/53584276 >> v1:172.21.15.38:6789/0 conn(0x7fd5c011fd60 legacy=0x7fd5c00653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:10.857 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 -- 172.21.15.38:0/53584276 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c010c050 con 0x7fd5c0091eb0 2020-03-19T23:11:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914358282 0 0) 0x7fd5c010af70 con 0x7fd5c011fd60 2020-03-19T23:11:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> v2:172.21.15.38:3300/0 conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> v2:172.21.15.38:3300/0 conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=READY pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd5b4005bc0 con 0x7fd5c0091eb0 2020-03-19T23:11:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.859 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.859 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b4005d20 con 0x7fd5c0091eb0 2020-03-19T23:11:10.859 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c0066c70 con 0x7fd5c0091eb0 2020-03-19T23:11:10.859 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd5b40061f0 con 0x7fd5c0091eb0 2020-03-19T23:11:10.859 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c008f450 con 0x7fd5c0091eb0 2020-03-19T23:11:10.860 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.847+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.860 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fd5b4006350 con 0x7fd5c0091eb0 2020-03-19T23:11:10.860 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.860 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5af7fe700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 0x7fd59c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:10.860 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fd5b4034110 con 0x7fd5c0091eb0 2020-03-19T23:11:10.861 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 0x7fd59c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:10.861 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 0x7fd59c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd5b00101c0 tx=0x7fd5b0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.861 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 0x7fd59c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd5b00101c0 tx=0x7fd5b0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.861 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5bdd9b700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 0x7fd59c02a740 secure :-1 s=READY pgs=40 cs=0 l=1 rx=0x7fd5b00101c0 tx=0x7fd5b0007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:10.861 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.848+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c011ce40 con 0x7fd5c0091eb0 2020-03-19T23:11:10.862 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.852+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:10.862 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:10.852+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fd5b4007ba0 con 0x7fd5c0091eb0 2020-03-19T23:11:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.001+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd5c0056db0 con 0x7fd5c0091eb0 2020-03-19T23:11:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.002+0000 7fd5be59c700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=THROTTLE_DONE pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.002+0000 7fd5af7fe700 1 -- 172.21.15.38:0/53584276 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5b4007160 con 0x7fd5c0091eb0 2020-03-19T23:11:11.004 INFO:teuthology.orchestra.run.smithi038.stdout:34359738372 2020-03-19T23:11:11.005 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 msgr2=0x7fd59c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 0x7fd59c02a740 secure :-1 s=READY pgs=40 cs=0 l=1 rx=0x7fd5b00101c0 tx=0x7fd5b0007a40).stop 2020-03-19T23:11:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 msgr2=0x7fd5c0064f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 secure :-1 s=READY pgs=157 cs=0 l=1 rx=0x7fd5b4000b80 tx=0x7fd5b4010930).stop 2020-03-19T23:11:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 shutdown_connections 2020-03-19T23:11:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c0091eb0 0x7fd5c0064f60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.006 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 --2- 172.21.15.38:0/53584276 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd59c028280 0x7fd59c02a740 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.007 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 >> 172.21.15.38:0/53584276 conn(0x7fd5c0091390 msgr2=0x7fd5c0060110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.007 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 shutdown_connections 2020-03-19T23:11:11.007 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.003+0000 7fd5c48b5700 1 -- 172.21.15.38:0/53584276 wait complete. 2020-03-19T23:11:11.015 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.0 2020-03-19T23:11:11.015 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-03-19T23:11:11.130 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.127+0000 7fc39fdac700 1 Processor -- start 2020-03-19T23:11:11.130 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.127+0000 7fc39fdac700 1 -- start start 2020-03-19T23:11:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39fdac700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398065cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39fdac700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3980604a0 con 0x7fc398066130 2020-03-19T23:11:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39fdac700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc398060810 con 0x7fc3980658f0 2020-03-19T23:11:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39db48700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398065cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39d347700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc398066130 0x7fc39808f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44034/0 (socket says 172.21.15.38:44034) 2020-03-19T23:11:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39d347700 1 -- 172.21.15.38:0/2352149254 learned_addr learned my addr 172.21.15.38:0/2352149254 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39db48700 1 -- 172.21.15.38:0/2352149254 >> v1:172.21.15.38:6789/0 conn(0x7fc398066130 legacy=0x7fc39808f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39db48700 1 -- 172.21.15.38:0/2352149254 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398060110 con 0x7fc3980658f0 2020-03-19T23:11:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39db48700 1 --2- 172.21.15.38:0/2352149254 >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398065cd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc3980615d0 tx=0x7fc38c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39db48700 1 --2- 172.21.15.38:0/2352149254 >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398065cd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc3980615d0 tx=0x7fc38c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.128+0000 7fc39db48700 1 --2- 172.21.15.38:0/2352149254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398065cd0 secure :-1 s=READY pgs=159 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c0088e0).ready entity=mon.0 client_cookie=2478860d37700f77 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39db48700 1 --2- 172.21.15.38:0/2352149254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398065cd0 secure :-1 s=THROTTLE_DONE pgs=159 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39db48700 1 --2- 172.21.15.38:0/2352149254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398065cd0 secure :-1 s=THROTTLE_DONE pgs=159 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39cb46700 1 -- 172.21.15.38:0/2352149254 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc38c0113f0 con 0x7fc3980658f0 2020-03-19T23:11:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39db48700 1 --2- 172.21.15.38:0/2352149254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398065cd0 secure :-1 s=THROTTLE_DONE pgs=159 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39cb46700 1 -- 172.21.15.38:0/2352149254 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c011550 con 0x7fc3980658f0 2020-03-19T23:11:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39cb46700 1 -- 172.21.15.38:0/2352149254 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc38c011870 con 0x7fc3980658f0 2020-03-19T23:11:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39fdac700 1 -- 172.21.15.38:0/2352149254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 msgr2=0x7fc398065cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39fdac700 1 --2- 172.21.15.38:0/2352149254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398065cd0 secure :-1 s=READY pgs=159 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c0088e0).stop 2020-03-19T23:11:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39fdac700 1 -- 172.21.15.38:0/2352149254 shutdown_connections 2020-03-19T23:11:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39fdac700 1 --2- 172.21.15.38:0/2352149254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398065cd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39fdac700 1 -- 172.21.15.38:0/2352149254 >> 172.21.15.38:0/2352149254 conn(0x7fc398091390 msgr2=0x7fc3980917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39fdac700 1 -- 172.21.15.38:0/2352149254 shutdown_connections 2020-03-19T23:11:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.129+0000 7fc39fdac700 1 -- 172.21.15.38:0/2352149254 wait complete. 2020-03-19T23:11:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39fdac700 1 Processor -- start 2020-03-19T23:11:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39fdac700 1 -- start start 2020-03-19T23:11:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39fdac700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39fdac700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc398130850 con 0x7fc398066130 2020-03-19T23:11:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39fdac700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc398111860 con 0x7fc3980658f0 2020-03-19T23:11:11.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39d347700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc398066130 0x7fc398130a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44038/0 (socket says 172.21.15.38:44038) 2020-03-19T23:11:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39d347700 1 -- 172.21.15.38:0/1401579010 learned_addr learned my addr 172.21.15.38:0/1401579010 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.130+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 -- 172.21.15.38:0/1401579010 >> v1:172.21.15.38:6789/0 conn(0x7fc398066130 legacy=0x7fc398130a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.141 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 -- 172.21.15.38:0/1401579010 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398113ea0 con 0x7fc3980658f0 2020-03-19T23:11:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> v2:172.21.15.38:3300/0 conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=READY pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc38affd700 1 -- 172.21.15.38:0/1401579010 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc38c00f7b0 con 0x7fc3980658f0 2020-03-19T23:11:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.150 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.150 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc38affd700 1 -- 172.21.15.38:0/1401579010 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c0056c0 con 0x7fc3980658f0 2020-03-19T23:11:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39810fe70 con 0x7fc3980658f0 2020-03-19T23:11:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc38affd700 1 -- 172.21.15.38:0/1401579010 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc38c0059a0 con 0x7fc3980658f0 2020-03-19T23:11:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc398131350 con 0x7fc3980658f0 2020-03-19T23:11:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.131+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.151 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc38affd700 1 -- 172.21.15.38:0/1401579010 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fc38c005c10 con 0x7fc3980658f0 2020-03-19T23:11:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc38affd700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 0x7fc37802a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc38affd700 1 -- 172.21.15.38:0/1401579010 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fc38c034750 con 0x7fc3980658f0 2020-03-19T23:11:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc39d347700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 0x7fc37802a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc398112ea0 con 0x7fc3980658f0 2020-03-19T23:11:11.152 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc39d347700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 0x7fc37802a7d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc39812e430 tx=0x7fc394001840).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc39d347700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 0x7fc37802a7d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc39812e430 tx=0x7fc394001840).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.132+0000 7fc39d347700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 0x7fc37802a7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7fc39812e430 tx=0x7fc394001840).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.136+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.154 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.136+0000 7fc38affd700 1 -- 172.21.15.38:0/1401579010 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fc38c014210 con 0x7fc3980658f0 2020-03-19T23:11:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.284+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc398056db0 con 0x7fc3980658f0 2020-03-19T23:11:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.284+0000 7fc39db48700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=THROTTLE_DONE pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.287 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.284+0000 7fc38affd700 1 -- 172.21.15.38:0/1401579010 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc38c009530 con 0x7fc3980658f0 2020-03-19T23:11:11.287 INFO:teuthology.orchestra.run.smithi038.stdout:34359738372 2020-03-19T23:11:11.288 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 msgr2=0x7fc37802a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.288 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 0x7fc37802a7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rx=0x7fc39812e430 tx=0x7fc394001840).stop 2020-03-19T23:11:11.288 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 msgr2=0x7fc398113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 secure :-1 s=READY pgs=161 cs=0 l=1 rx=0x7fc3980615d0 tx=0x7fc38c011390).stop 2020-03-19T23:11:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 shutdown_connections 2020-03-19T23:11:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3980658f0 0x7fc398113700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 --2- 172.21.15.38:0/1401579010 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc378028310 0x7fc37802a7d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 >> 172.21.15.38:0/1401579010 conn(0x7fc398091390 msgr2=0x7fc39808d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 shutdown_connections 2020-03-19T23:11:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.285+0000 7fc39fdac700 1 -- 172.21.15.38:0/1401579010 wait complete. 2020-03-19T23:11:11.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.1 2020-03-19T23:11:11.298 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-03-19T23:11:11.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9da54700 1 Processor -- start 2020-03-19T23:11:11.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9da54700 1 -- start start 2020-03-19T23:11:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9da54700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e9811f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9da54700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e98090e50 con 0x7f9e9811fcb0 2020-03-19T23:11:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9da54700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e98091190 con 0x7f9e98061310 2020-03-19T23:11:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9ca52700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e9811f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9ca52700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e9811f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60904/0 (socket says 172.21.15.38:60904) 2020-03-19T23:11:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e97fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9e9811fcb0 0x7f9e98060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44046/0 (socket says 172.21.15.38:44046) 2020-03-19T23:11:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9ca52700 1 -- 172.21.15.38:0/1844264619 learned_addr learned my addr 172.21.15.38:0/1844264619 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9ca52700 1 -- 172.21.15.38:0/1844264619 >> v1:172.21.15.38:6789/0 conn(0x7f9e9811fcb0 legacy=0x7f9e98060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.411+0000 7f9e9ca52700 1 -- 172.21.15.38:0/1844264619 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9811f540 con 0x7f9e98061310 2020-03-19T23:11:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/1844264619 >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e9811f850 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f9e980612d0 tx=0x7f9e880088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/1844264619 >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e9811f850 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f9e980612d0 tx=0x7f9e880088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/1844264619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e9811f850 secure :-1 s=READY pgs=163 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e880088e0).ready entity=mon.0 client_cookie=f6fe6389127bb017 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/1844264619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e9811f850 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e880088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e977fe700 1 -- 172.21.15.38:0/1844264619 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9e88013c20 con 0x7f9e98061310 2020-03-19T23:11:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/1844264619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e9811f850 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e880088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/1844264619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e9811f850 secure :-1 s=THROTTLE_DONE pgs=163 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e880088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e977fe700 1 -- 172.21.15.38:0/1844264619 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e88013d80 con 0x7f9e98061310 2020-03-19T23:11:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e977fe700 1 -- 172.21.15.38:0/1844264619 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9e8800c290 con 0x7f9e98061310 2020-03-19T23:11:11.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9da54700 1 -- 172.21.15.38:0/1844264619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 msgr2=0x7f9e9811f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9da54700 1 --2- 172.21.15.38:0/1844264619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e9811f850 secure :-1 s=READY pgs=163 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e880088e0).stop 2020-03-19T23:11:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9da54700 1 -- 172.21.15.38:0/1844264619 shutdown_connections 2020-03-19T23:11:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9da54700 1 --2- 172.21.15.38:0/1844264619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e9811f850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9da54700 1 -- 172.21.15.38:0/1844264619 >> 172.21.15.38:0/1844264619 conn(0x7f9e98091390 msgr2=0x7f9e98091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9da54700 1 -- 172.21.15.38:0/1844264619 shutdown_connections 2020-03-19T23:11:11.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.412+0000 7f9e9da54700 1 -- 172.21.15.38:0/1844264619 wait complete. 2020-03-19T23:11:11.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.413+0000 7f9e9da54700 1 Processor -- start 2020-03-19T23:11:11.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.413+0000 7f9e9da54700 1 -- start start 2020-03-19T23:11:11.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9da54700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e98066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9da54700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9811c860 con 0x7f9e9811fcb0 2020-03-19T23:11:11.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9da54700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e98063350 con 0x7f9e98061310 2020-03-19T23:11:11.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e98066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e98066a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60908/0 (socket says 172.21.15.38:60908) 2020-03-19T23:11:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 -- 172.21.15.38:0/2402525824 learned_addr learned my addr 172.21.15.38:0/2402525824 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 -- 172.21.15.38:0/2402525824 >> v1:172.21.15.38:6789/0 conn(0x7f9e9811fcb0 legacy=0x7f9e9811bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551041321 0 0) 0x7f9e9811c860 con 0x7f9e9811fcb0 2020-03-19T23:11:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 -- 172.21.15.38:0/2402525824 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9810b7c0 con 0x7f9e98061310 2020-03-19T23:11:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> v2:172.21.15.38:3300/0 conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=READY pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9e88007280 con 0x7f9e98061310 2020-03-19T23:11:11.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e880073e0 con 0x7f9e98061310 2020-03-19T23:11:11.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9810b4b0 con 0x7f9e98061310 2020-03-19T23:11:11.426 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9e8800c710 con 0x7f9e98061310 2020-03-19T23:11:11.426 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.414+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9811c1d0 con 0x7f9e98061310 2020-03-19T23:11:11.426 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.426 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f9e8800c910 con 0x7f9e98061310 2020-03-19T23:11:11.426 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.426 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e95ffb700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 0x7f9e7802a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.426 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f9e88036f90 con 0x7f9e98061310 2020-03-19T23:11:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e97fff700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 0x7f9e7802a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9811c6a0 con 0x7f9e98061310 2020-03-19T23:11:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e97fff700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 0x7f9e7802a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9e8c001160 tx=0x7f9e8c0016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e97fff700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 0x7f9e7802a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9e8c001160 tx=0x7f9e8c0016c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.415+0000 7f9e97fff700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 0x7f9e7802a6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rx=0x7f9e8c001160 tx=0x7f9e8c0016c0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.419+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.428 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.419+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f9e9811c6a0 con 0x7f9e98061310 2020-03-19T23:11:11.570 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.568+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e9811c6a0 con 0x7f9e98061310 2020-03-19T23:11:11.570 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.568+0000 7f9e9ca52700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=THROTTLE_DONE pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.570 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.568+0000 7f9e95ffb700 1 -- 172.21.15.38:0/2402525824 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e9811c6a0 con 0x7f9e98061310 2020-03-19T23:11:11.571 INFO:teuthology.orchestra.run.smithi038.stdout:34359738373 2020-03-19T23:11:11.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 msgr2=0x7f9e7802a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 0x7f9e7802a6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rx=0x7f9e8c001160 tx=0x7f9e8c0016c0).stop 2020-03-19T23:11:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 msgr2=0x7f9e98066a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 secure :-1 s=READY pgs=165 cs=0 l=1 rx=0x7f9e980612d0 tx=0x7f9e9811f6d0).stop 2020-03-19T23:11:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 shutdown_connections 2020-03-19T23:11:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e98061310 0x7f9e98066a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 --2- 172.21.15.38:0/2402525824 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f9e78028200 0x7f9e7802a6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.573 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 >> 172.21.15.38:0/2402525824 conn(0x7f9e98091390 msgr2=0x7f9e980931b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.573 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 shutdown_connections 2020-03-19T23:11:11.573 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.569+0000 7f9e9da54700 1 -- 172.21.15.38:0/2402525824 wait complete. 2020-03-19T23:11:11.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2020-03-19T23:11:11.582 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-03-19T23:11:11.582 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T23:11:11.695 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e24d04700 1 Processor -- start 2020-03-19T23:11:11.695 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e24d04700 1 -- start start 2020-03-19T23:11:11.700 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e24d04700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e200907c0 0x7f7e2008daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:11.700 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e24d04700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e200605e0 con 0x7f7e2008d000 2020-03-19T23:11:11.700 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e24d04700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2008ef70 con 0x7f7e200907c0 2020-03-19T23:11:11.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e1dd9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e200907c0 0x7f7e2008daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e1e59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7e2008d000 0x7f7e2008d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44054/0 (socket says 172.21.15.38:44054) 2020-03-19T23:11:11.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e1dd9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e200907c0 0x7f7e2008daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60916/0 (socket says 172.21.15.38:60916) 2020-03-19T23:11:11.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.693+0000 7f7e1e59c700 1 -- 172.21.15.38:0/1697788189 learned_addr learned my addr 172.21.15.38:0/1697788189 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:11.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 -- 172.21.15.38:0/1697788189 >> v1:172.21.15.38:6789/0 conn(0x7f7e2008d000 legacy=0x7f7e2008d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1d59a700 1 -- 172.21.15.38:0/1697788189 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077781951 0 0) 0x7f7e200605e0 con 0x7f7e2008d000 2020-03-19T23:11:11.701 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 -- 172.21.15.38:0/1697788189 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20060810 con 0x7f7e200907c0 2020-03-19T23:11:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1697788189 >> v2:172.21.15.38:3300/0 conn(0x7f7e200907c0 0x7f7e2008daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7e20090c70 tx=0x7f7e08008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1697788189 >> v2:172.21.15.38:3300/0 conn(0x7f7e200907c0 0x7f7e2008daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7e20090c70 tx=0x7f7e08008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1697788189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e200907c0 0x7f7e2008daf0 secure :-1 s=READY pgs=167 cs=0 l=1 rx=0x7f7e20090c70 tx=0x7f7e08008c60).ready entity=mon.0 client_cookie=567f0d4f848ae6cd server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1697788189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e200907c0 0x7f7e2008daf0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f7e20090c70 tx=0x7f7e08008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1d59a700 1 -- 172.21.15.38:0/1697788189 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e08011cd0 con 0x7f7e200907c0 2020-03-19T23:11:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1697788189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e200907c0 0x7f7e2008daf0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f7e20090c70 tx=0x7f7e08008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1697788189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e200907c0 0x7f7e2008daf0 secure :-1 s=THROTTLE_DONE pgs=167 cs=0 l=1 rx=0x7f7e20090c70 tx=0x7f7e08008c60).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.703 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1d59a700 1 -- 172.21.15.38:0/1697788189 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e08011e30 con 0x7f7e200907c0 2020-03-19T23:11:11.703 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e1d59a700 1 -- 172.21.15.38:0/1697788189 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e0800a290 con 0x7f7e200907c0 2020-03-19T23:11:11.703 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e24d04700 1 -- 172.21.15.38:0/1697788189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e200907c0 msgr2=0x7f7e2008daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.703 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.694+0000 7f7e24d04700 1 --2- 172.21.15.38:0/1697788189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e200907c0 0x7f7e2008daf0 secure :-1 s=READY pgs=167 cs=0 l=1 rx=0x7f7e20090c70 tx=0x7f7e08008c60).stop 2020-03-19T23:11:11.703 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.695+0000 7f7e24d04700 1 -- 172.21.15.38:0/1697788189 shutdown_connections 2020-03-19T23:11:11.703 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.695+0000 7f7e24d04700 1 --2- 172.21.15.38:0/1697788189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e200907c0 0x7f7e2008daf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.695+0000 7f7e24d04700 1 -- 172.21.15.38:0/1697788189 >> 172.21.15.38:0/1697788189 conn(0x7f7e20091540 msgr2=0x7f7e20091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.695+0000 7f7e24d04700 1 -- 172.21.15.38:0/1697788189 shutdown_connections 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.695+0000 7f7e24d04700 1 -- 172.21.15.38:0/1697788189 wait complete. 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.695+0000 7f7e24d04700 1 Processor -- start 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.695+0000 7f7e24d04700 1 -- start start 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e24d04700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e2008d000 0x7f7e2011f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e24d04700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2011fec0 con 0x7f7e200907c0 2020-03-19T23:11:11.704 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e24d04700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2011bca0 con 0x7f7e2008d000 2020-03-19T23:11:11.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e2008d000 0x7f7e2011f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e2008d000 0x7f7e2011f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60918/0 (socket says 172.21.15.38:60918) 2020-03-19T23:11:11.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 -- 172.21.15.38:0/1745177601 learned_addr learned my addr 172.21.15.38:0/1745177601 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:11.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1dd9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7e200907c0 0x7f7e20122310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44060/0 (socket says 172.21.15.38:44060) 2020-03-19T23:11:11.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 -- 172.21.15.38:0/1745177601 >> v1:172.21.15.38:6789/0 conn(0x7f7e200907c0 legacy=0x7f7e20122310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 -- 172.21.15.38:0/1745177601 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2011e910 con 0x7f7e2008d000 2020-03-19T23:11:11.705 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425857203 0 0) 0x7f7e2011fec0 con 0x7f7e200907c0 2020-03-19T23:11:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> v2:172.21.15.38:3300/0 conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> v2:172.21.15.38:3300/0 conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=READY pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e18005b70 con 0x7f7e2008d000 2020-03-19T23:11:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.706 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e18005cd0 con 0x7f7e2008d000 2020-03-19T23:11:11.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2011fbb0 con 0x7f7e2008d000 2020-03-19T23:11:11.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.696+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e180060c0 con 0x7f7e2008d000 2020-03-19T23:11:11.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.697+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e20122a20 con 0x7f7e2008d000 2020-03-19T23:11:11.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.697+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.697+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f7e18006330 con 0x7f7e2008d000 2020-03-19T23:11:11.707 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.697+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.697+0000 7f7e177fe700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.697+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f7e180340b0 con 0x7f7e2008d000 2020-03-19T23:11:11.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.697+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.698+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7e08010540 tx=0x7f7e20067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.698+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7e08010540 tx=0x7f7e20067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.698+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 secure :-1 s=READY pgs=43 cs=0 l=1 rx=0x7f7e08010540 tx=0x7f7e20067850).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.708 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.698+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e20056db0 con 0x7f7e2008d000 2020-03-19T23:11:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.701+0000 7f7e1e59c700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=THROTTLE_DONE pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.701+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f7e18007340 con 0x7f7e2008d000 2020-03-19T23:11:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.844+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e2011e230 con 0x7f7e04028250 2020-03-19T23:11:11.849 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.847+0000 7f7e1dd9b700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 secure :-1 s=THROTTLE_DONE pgs=43 cs=0 l=1 rx=0x7f7e08010540 tx=0x7f7e20067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.847+0000 7f7e177fe700 1 -- 172.21.15.38:0/1745177601 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+63952 (secure 0 0 0) 0x7f7e0800a160 con 0x7f7e04028250 2020-03-19T23:11:11.850 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 msgr2=0x7f7e0402a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 secure :-1 s=READY pgs=43 cs=0 l=1 rx=0x7f7e08010540 tx=0x7f7e20067850).stop 2020-03-19T23:11:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 msgr2=0x7f7e2011f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 secure :-1 s=READY pgs=169 cs=0 l=1 rx=0x7f7e2011f130 tx=0x7f7e18010990).stop 2020-03-19T23:11:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 shutdown_connections 2020-03-19T23:11:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e2008d000 0x7f7e2011f250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 --2- 172.21.15.38:0/1745177601 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e04028250 0x7f7e0402a710 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 >> 172.21.15.38:0/1745177601 conn(0x7f7e20091540 msgr2=0x7f7e200629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 shutdown_connections 2020-03-19T23:11:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.848+0000 7f7e24d04700 1 -- 172.21.15.38:0/1745177601 wait complete. 2020-03-19T23:11:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2020-03-19T23:11:11.860 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-03-19T23:11:11.616446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146224,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426576,"statfs":{"total":322122547200,"available":318900813824,"internally_reserved":3221225472,"allocated":483328,"data_stored":132129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000331"},"pg_stats":[{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.654160+0000","last_change":"2020-03-19T23:11:04.653879+0000","last_active":"2020-03-19T23:11:04.654160+0000","last_peered":"2020-03-19T23:11:04.654160+0000","last_clean":"2020-03-19T23:11:04.654160+0000","last_became_active":"2020-03-19T23:11:04.653521+0000","last_became_peered":"2020-03-19T23:11:04.653521+0000","last_unstale":"2020-03-19T23:11:04.654160+0000","last_undegraded":"2020-03-19T23:11:04.654160+0000","last_fullsized":"2020-03-19T23:11:04.654160+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.648376+0000","last_change":"2020-03-19T23:11:01.250472+0000","last_active":"2020-03-19T23:11:04.648376+0000","last_peered":"2020-03-19T23:11:04.648376+0000","last_clean":"2020-03-19T23:11:04.648376+0000","last_became_active":"2020-03-19T23:11:01.250084+0000","last_became_peered":"2020-03-19T23:11:01.250084+0000","last_unstale":"2020-03-19T23:11:04.648376+0000","last_undegraded":"2020-03-19T23:11:04.648376+0000","last_fullsized":"2020-03-19T23:11:04.648376+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'3","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.324272+0000","last_change":"2020-03-19T23:11:04.285615+0000","last_active":"2020-03-19T23:11:07.324272+0000","last_peered":"2020-03-19T23:11:07.324272+0000","last_clean":"2020-03-19T23:11:07.324272+0000","last_became_active":"2020-03-19T23:11:04.284829+0000","last_became_peered":"2020-03-19T23:11:04.284829+0000","last_unstale":"2020-03-19T23:11:07.324272+0000","last_undegraded":"2020-03-19T23:11:07.324272+0000","last_fullsized":"2020-03-19T23:11:07.324272+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.654566+0000","last_change":"2020-03-19T23:11:04.654105+0000","last_active":"2020-03-19T23:11:04.654566+0000","last_peered":"2020-03-19T23:11:04.654566+0000","last_clean":"2020-03-19T23:11:04.654566+0000","last_became_active":"2020-03-19T23:11:04.653741+0000","last_became_peered":"2020-03-19T23:11:04.653741+0000","last_unstale":"2020-03-19T23:11:04.654566+0000","last_undegraded":"2020-03-19T23:11:04.654566+0000","last_fullsized":"2020-03-19T23:11:04.654566+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.647891+0000","last_change":"2020-03-19T23:11:01.251743+0000","last_active":"2020-03-19T23:11:04.647891+0000","last_peered":"2020-03-19T23:11:04.647891+0000","last_clean":"2020-03-19T23:11:04.647891+0000","last_became_active":"2020-03-19T23:11:01.251378+0000","last_became_peered":"2020-03-19T23:11:01.251378+0000","last_unstale":"2020-03-19T23:11:04.647891+0000","last_undegraded":"2020-03-19T23:11:04.647891+0000","last_fullsized":"2020-03-19T23:11:04.647891+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"17","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.324921+0000","last_change":"2020-03-19T23:11:04.285389+0000","last_active":"2020-03-19T23:11:07.324921+0000","last_peered":"2020-03-19T23:11:07.324921+0000","last_clean":"2020-03-19T23:11:07.324921+0000","last_became_active":"2020-03-19T23:11:04.284979+0000","last_became_peered":"2020-03-19T23:11:04.284979+0000","last_unstale":"2020-03-19T23:11:07.324921+0000","last_undegraded":"2020-03-19T23:11:07.324921+0000","last_fullsized":"2020-03-19T23:11:07.324921+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653068+0000","last_change":"2020-03-19T23:11:04.650535+0000","last_active":"2020-03-19T23:11:04.653068+0000","last_peered":"2020-03-19T23:11:04.653068+0000","last_clean":"2020-03-19T23:11:04.653068+0000","last_became_active":"2020-03-19T23:11:04.649316+0000","last_became_peered":"2020-03-19T23:11:04.649316+0000","last_unstale":"2020-03-19T23:11:04.653068+0000","last_undegraded":"2020-03-19T23:11:04.653068+0000","last_fullsized":"2020-03-19T23:11:04.653068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.647577+0000","last_change":"2020-03-19T23:11:01.248950+0000","last_active":"2020-03-19T23:11:04.647577+0000","last_peered":"2020-03-19T23:11:04.647577+0000","last_clean":"2020-03-19T23:11:04.647577+0000","last_became_active":"2020-03-19T23:11:01.248513+0000","last_became_peered":"2020-03-19T23:11:01.248513+0000","last_unstale":"2020-03-19T23:11:04.647577+0000","last_undegraded":"2020-03-19T23:11:04.647577+0000","last_fullsized":"2020-03-19T23:11:04.647577+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.319991+0000","last_change":"2020-03-19T23:11:04.288163+0000","last_active":"2020-03-19T23:11:07.319991+0000","last_peered":"2020-03-19T23:11:07.319991+0000","last_clean":"2020-03-19T23:11:07.319991+0000","last_became_active":"2020-03-19T23:11:04.287139+0000","last_became_peered":"2020-03-19T23:11:04.287139+0000","last_unstale":"2020-03-19T23:11:07.319991+0000","last_undegraded":"2020-03-19T23:11:07.319991+0000","last_fullsized":"2020-03-19T23:11:07.319991+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.654283+0000","last_change":"2020-03-19T23:11:04.654111+0000","last_active":"2020-03-19T23:11:04.654283+0000","last_peered":"2020-03-19T23:11:04.654283+0000","last_clean":"2020-03-19T23:11:04.654283+0000","last_became_active":"2020-03-19T23:11:04.653768+0000","last_became_peered":"2020-03-19T23:11:04.653768+0000","last_unstale":"2020-03-19T23:11:04.654283+0000","last_undegraded":"2020-03-19T23:11:04.654283+0000","last_fullsized":"2020-03-19T23:11:04.654283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.645773+0000","last_change":"2020-03-19T23:11:01.252367+0000","last_active":"2020-03-19T23:11:04.645773+0000","last_peered":"2020-03-19T23:11:04.645773+0000","last_clean":"2020-03-19T23:11:04.645773+0000","last_became_active":"2020-03-19T23:11:01.251474+0000","last_became_peered":"2020-03-19T23:11:01.251474+0000","last_unstale":"2020-03-19T23:11:04.645773+0000","last_undegraded":"2020-03-19T23:11:04.645773+0000","last_fullsized":"2020-03-19T23:11:04.645773+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.648269+0000","last_change":"2020-03-19T23:11:04.287593+0000","last_active":"2020-03-19T23:11:04.648269+0000","last_peered":"2020-03-19T23:11:04.648269+0000","last_clean":"2020-03-19T23:11:04.648269+0000","last_became_active":"2020-03-19T23:11:04.286797+0000","last_became_peered":"2020-03-19T23:11:04.286797+0000","last_unstale":"2020-03-19T23:11:04.648269+0000","last_undegraded":"2020-03-19T23:11:04.648269+0000","last_fullsized":"2020-03-19T23:11:04.648269+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653128+0000","last_change":"2020-03-19T23:11:04.650882+0000","last_active":"2020-03-19T23:11:04.653128+0000","last_peered":"2020-03-19T23:11:04.653128+0000","last_clean":"2020-03-19T23:11:04.653128+0000","last_became_active":"2020-03-19T23:11:04.650546+0000","last_became_peered":"2020-03-19T23:11:04.650546+0000","last_unstale":"2020-03-19T23:11:04.653128+0000","last_undegraded":"2020-03-19T23:11:04.653128+0000","last_fullsized":"2020-03-19T23:11:04.653128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.649563+0000","last_change":"2020-03-19T23:11:01.249741+0000","last_active":"2020-03-19T23:11:04.649563+0000","last_peered":"2020-03-19T23:11:04.649563+0000","last_clean":"2020-03-19T23:11:04.649563+0000","last_became_active":"2020-03-19T23:11:01.249356+0000","last_became_peered":"2020-03-19T23:11:01.249356+0000","last_unstale":"2020-03-19T23:11:04.649563+0000","last_undegraded":"2020-03-19T23:11:04.649563+0000","last_fullsized":"2020-03-19T23:11:04.649563+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.324070+0000","last_change":"2020-03-19T23:11:04.284232+0000","last_active":"2020-03-19T23:11:07.324070+0000","last_peered":"2020-03-19T23:11:07.324070+0000","last_clean":"2020-03-19T23:11:07.324070+0000","last_became_active":"2020-03-19T23:11:04.283884+0000","last_became_peered":"2020-03-19T23:11:04.283884+0000","last_unstale":"2020-03-19T23:11:07.324070+0000","last_undegraded":"2020-03-19T23:11:07.324070+0000","last_fullsized":"2020-03-19T23:11:07.324070+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653060+0000","last_change":"2020-03-19T23:11:04.652133+0000","last_active":"2020-03-19T23:11:04.653060+0000","last_peered":"2020-03-19T23:11:04.653060+0000","last_clean":"2020-03-19T23:11:04.653060+0000","last_became_active":"2020-03-19T23:11:04.651761+0000","last_became_peered":"2020-03-19T23:11:04.651761+0000","last_unstale":"2020-03-19T23:11:04.653060+0000","last_undegraded":"2020-03-19T23:11:04.653060+0000","last_fullsized":"2020-03-19T23:11:04.653060+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.648086+0000","last_change":"2020-03-19T23:11:01.250680+0000","last_active":"2020-03-19T23:11:04.648086+0000","last_peered":"2020-03-19T23:11:04.648086+0000","last_clean":"2020-03-19T23:11:04.648086+0000","last_became_active":"2020-03-19T23:11:01.250171+0000","last_became_peered":"2020-03-19T23:11:01.250171+0000","last_unstale":"2020-03-19T23:11:04.648086+0000","last_undegraded":"2020-03-19T23:11:04.648086+0000","last_fullsized":"2020-03-19T23:11:04.648086+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:05.645359+0000","last_change":"2020-03-19T23:11:04.287858+0000","last_active":"2020-03-19T23:11:05.645359+0000","last_peered":"2020-03-19T23:11:05.645359+0000","last_clean":"2020-03-19T23:11:05.645359+0000","last_became_active":"2020-03-19T23:11:04.287457+0000","last_became_peered":"2020-03-19T23:11:04.287457+0000","last_unstale":"2020-03-19T23:11:05.645359+0000","last_undegraded":"2020-03-19T23:11:05.645359+0000","last_fullsized":"2020-03-19T23:11:05.645359+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.655287+0000","last_change":"2020-03-19T23:11:04.654429+0000","last_active":"2020-03-19T23:11:04.655287+0000","last_peered":"2020-03-19T23:11:04.655287+0000","last_clean":"2020-03-19T23:11:04.655287+0000","last_became_active":"2020-03-19T23:11:04.654078+0000","last_became_peered":"2020-03-19T23:11:04.654078+0000","last_unstale":"2020-03-19T23:11:04.655287+0000","last_undegraded":"2020-03-19T23:11:04.655287+0000","last_fullsized":"2020-03-19T23:11:04.655287+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.649254+0000","last_change":"2020-03-19T23:11:01.252623+0000","last_active":"2020-03-19T23:11:04.649254+0000","last_peered":"2020-03-19T23:11:04.649254+0000","last_clean":"2020-03-19T23:11:04.649254+0000","last_became_active":"2020-03-19T23:11:01.252205+0000","last_became_peered":"2020-03-19T23:11:01.252205+0000","last_unstale":"2020-03-19T23:11:04.649254+0000","last_undegraded":"2020-03-19T23:11:04.649254+0000","last_fullsized":"2020-03-19T23:11:04.649254+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.325527+0000","last_change":"2020-03-19T23:11:04.285188+0000","last_active":"2020-03-19T23:11:07.325527+0000","last_peered":"2020-03-19T23:11:07.325527+0000","last_clean":"2020-03-19T23:11:07.325527+0000","last_became_active":"2020-03-19T23:11:04.284579+0000","last_became_peered":"2020-03-19T23:11:04.284579+0000","last_unstale":"2020-03-19T23:11:07.325527+0000","last_undegraded":"2020-03-19T23:11:07.325527+0000","last_fullsized":"2020-03-19T23:11:07.325527+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653327+0000","last_change":"2020-03-19T23:11:04.650468+0000","last_active":"2020-03-19T23:11:04.653327+0000","last_peered":"2020-03-19T23:11:04.653327+0000","last_clean":"2020-03-19T23:11:04.653327+0000","last_became_active":"2020-03-19T23:11:04.650124+0000","last_became_peered":"2020-03-19T23:11:04.650124+0000","last_unstale":"2020-03-19T23:11:04.653327+0000","last_undegraded":"2020-03-19T23:11:04.653327+0000","last_fullsized":"2020-03-19T23:11:04.653327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.647588+0000","last_change":"2020-03-19T23:11:01.249394+0000","last_active":"2020-03-19T23:11:04.647588+0000","last_peered":"2020-03-19T23:11:04.647588+0000","last_clean":"2020-03-19T23:11:04.647588+0000","last_became_active":"2020-03-19T23:11:01.249024+0000","last_became_peered":"2020-03-19T23:11:01.249024+0000","last_unstale":"2020-03-19T23:11:04.647588+0000","last_undegraded":"2020-03-19T23:11:04.647588+0000","last_fullsized":"2020-03-19T23:11:04.647588+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.325123+0000","last_change":"2020-03-19T23:11:04.283517+0000","last_active":"2020-03-19T23:11:07.325123+0000","last_peered":"2020-03-19T23:11:07.325123+0000","last_clean":"2020-03-19T23:11:07.325123+0000","last_became_active":"2020-03-19T23:11:04.283065+0000","last_became_peered":"2020-03-19T23:11:04.283065+0000","last_unstale":"2020-03-19T23:11:07.325123+0000","last_undegraded":"2020-03-19T23:11:07.325123+0000","last_fullsized":"2020-03-19T23:11:07.325123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048724,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808876,"statfs":{"total":107374182400,"available":106300289024,"internally_reserved":1073741824,"allocated":143360,"data_stored":42553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048748,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808852,"statfs":{"total":107374182400,"available":106300264448,"internally_reserved":1073741824,"allocated":167936,"data_stored":44771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048752,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808848,"statfs":{"total":107374182400,"available":106300260352,"internally_reserved":1073741824,"allocated":172032,"data_stored":44805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]}]}} 2020-03-19T23:11:11.864 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T23:11:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.975+0000 7fcfa0e95700 1 Processor -- start 2020-03-19T23:11:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.975+0000 7fcfa0e95700 1 -- start start 2020-03-19T23:11:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.975+0000 7fcfa0e95700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c08daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.975+0000 7fcfa0e95700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9c0605e0 con 0x7fcf9c08d000 2020-03-19T23:11:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.975+0000 7fcfa0e95700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9c08ef70 con 0x7fcf9c0907c0 2020-03-19T23:11:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c08daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c08daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60924/0 (socket says 172.21.15.38:60924) 2020-03-19T23:11:11.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf9a59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcf9c08d000 0x7fcf9c08d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44064/0 (socket says 172.21.15.38:44064) 2020-03-19T23:11:11.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 -- 172.21.15.38:0/1588258793 learned_addr learned my addr 172.21.15.38:0/1588258793 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:11.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 -- 172.21.15.38:0/1588258793 >> v1:172.21.15.38:6789/0 conn(0x7fcf9c08d000 legacy=0x7fcf9c08d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 -- 172.21.15.38:0/1588258793 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c060810 con 0x7fcf9c0907c0 2020-03-19T23:11:11.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf9959a700 1 -- 172.21.15.38:0/1588258793 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805805380 0 0) 0x7fcf9c0605e0 con 0x7fcf9c08d000 2020-03-19T23:11:11.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1588258793 >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c08daf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fcf9c090c70 tx=0x7fcf8c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1588258793 >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c08daf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fcf9c090c70 tx=0x7fcf8c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1588258793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c08daf0 secure :-1 s=READY pgs=171 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf8c0088e0).ready entity=mon.0 client_cookie=d956e0c7c826c13d server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1588258793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c08daf0 secure :-1 s=THROTTLE_DONE pgs=171 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf8c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf9959a700 1 -- 172.21.15.38:0/1588258793 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcf8c013c20 con 0x7fcf9c0907c0 2020-03-19T23:11:11.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1588258793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c08daf0 secure :-1 s=THROTTLE_DONE pgs=171 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf8c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1588258793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c08daf0 secure :-1 s=THROTTLE_DONE pgs=171 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf8c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf9959a700 1 -- 172.21.15.38:0/1588258793 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c013d80 con 0x7fcf9c0907c0 2020-03-19T23:11:11.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcf9959a700 1 -- 172.21.15.38:0/1588258793 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcf8c00c290 con 0x7fcf9c0907c0 2020-03-19T23:11:11.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1588258793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 msgr2=0x7fcf9c08daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.976+0000 7fcfa0e95700 1 --2- 172.21.15.38:0/1588258793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c08daf0 secure :-1 s=READY pgs=171 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf8c0088e0).stop 2020-03-19T23:11:11.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.977+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1588258793 shutdown_connections 2020-03-19T23:11:11.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.977+0000 7fcfa0e95700 1 --2- 172.21.15.38:0/1588258793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c08daf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:11.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.977+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1588258793 >> 172.21.15.38:0/1588258793 conn(0x7fcf9c091540 msgr2=0x7fcf9c091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:11.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.977+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1588258793 shutdown_connections 2020-03-19T23:11:11.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.977+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1588258793 wait complete. 2020-03-19T23:11:11.986 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.977+0000 7fcfa0e95700 1 Processor -- start 2020-03-19T23:11:11.986 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.977+0000 7fcfa0e95700 1 -- start start 2020-03-19T23:11:11.986 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcfa0e95700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c11bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.986 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcfa0e95700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9c11dbf0 con 0x7fcf9c08d000 2020-03-19T23:11:11.986 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcfa0e95700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9c11d480 con 0x7fcf9c0907c0 2020-03-19T23:11:11.986 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf9a59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcf9c08d000 0x7fcf9c122470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44068/0 (socket says 172.21.15.38:44068) 2020-03-19T23:11:11.986 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c11bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf9a59c700 1 -- 172.21.15.38:0/1604263967 learned_addr learned my addr 172.21.15.38:0/1604263967 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:11.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 -- 172.21.15.38:0/1604263967 >> v1:172.21.15.38:6789/0 conn(0x7fcf9c08d000 legacy=0x7fcf9c122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:11.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 -- 172.21.15.38:0/1604263967 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c11f240 con 0x7fcf9c0907c0 2020-03-19T23:11:11.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502976247 0 0) 0x7fcf9c11dbf0 con 0x7fcf9c08d000 2020-03-19T23:11:11.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> v2:172.21.15.38:3300/0 conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=READY pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcf8c007150 con 0x7fcf9c0907c0 2020-03-19T23:11:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.978+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c0072b0 con 0x7fcf9c0907c0 2020-03-19T23:11:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fcf8c007590 con 0x7fcf9c0907c0 2020-03-19T23:11:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9c11ef90 con 0x7fcf9c0907c0 2020-03-19T23:11:11.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9c122b80 con 0x7fcf9c0907c0 2020-03-19T23:11:11.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fcf8c005710 con 0x7fcf9c0907c0 2020-03-19T23:11:11.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcf8b7fe700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:11.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fcf8c036e70 con 0x7fcf9c0907c0 2020-03-19T23:11:11.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.979+0000 7fcf9a59c700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:11.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.980+0000 7fcf9a59c700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fcf90001170 tx=0x7fcf90001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.980+0000 7fcf9a59c700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fcf90001170 tx=0x7fcf90001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.980+0000 7fcf9a59c700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 secure :-1 s=READY pgs=44 cs=0 l=1 rx=0x7fcf90001170 tx=0x7fcf90001780).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:11.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.980+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9c11dd80 con 0x7fcf9c0907c0 2020-03-19T23:11:11.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.983+0000 7fcf99d9b700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=THROTTLE_DONE pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:11.991 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:11.983+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fcf8c0059a0 con 0x7fcf9c0907c0 2020-03-19T23:11:12.129 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.127+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf9c056db0 con 0x7fcf78028230 2020-03-19T23:11:12.131 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.129+0000 7fcf9a59c700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 secure :-1 s=THROTTLE_DONE pgs=44 cs=0 l=1 rx=0x7fcf90001170 tx=0x7fcf90001780).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.132 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.129+0000 7fcf8b7fe700 1 -- 172.21.15.38:0/1604263967 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+63952 (secure 0 0 0) 0x7fcf9c056db0 con 0x7fcf78028230 2020-03-19T23:11:12.132 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:12.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 msgr2=0x7fcf7802a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 secure :-1 s=READY pgs=44 cs=0 l=1 rx=0x7fcf90001170 tx=0x7fcf90001780).stop 2020-03-19T23:11:12.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 msgr2=0x7fcf9c11bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 secure :-1 s=READY pgs=173 cs=0 l=1 rx=0x7fcf9c090c70 tx=0x7fcf9c067850).stop 2020-03-19T23:11:12.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 shutdown_connections 2020-03-19T23:11:12.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fcf78028230 0x7fcf7802a6f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 --2- 172.21.15.38:0/1604263967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf9c0907c0 0x7fcf9c11bb00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 >> 172.21.15.38:0/1604263967 conn(0x7fcf9c091540 msgr2=0x7fcf9c093620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:12.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 shutdown_connections 2020-03-19T23:11:12.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.130+0000 7fcfa0e95700 1 -- 172.21.15.38:0/1604263967 wait complete. 2020-03-19T23:11:12.135 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2020-03-19T23:11:12.143 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2020-03-19T23:11:11.616446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146224,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426576,"statfs":{"total":322122547200,"available":318900813824,"internally_reserved":3221225472,"allocated":483328,"data_stored":132129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000331"},"pg_stats":[{"pgid":"3.7","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.654160+0000","last_change":"2020-03-19T23:11:04.653879+0000","last_active":"2020-03-19T23:11:04.654160+0000","last_peered":"2020-03-19T23:11:04.654160+0000","last_clean":"2020-03-19T23:11:04.654160+0000","last_became_active":"2020-03-19T23:11:04.653521+0000","last_became_peered":"2020-03-19T23:11:04.653521+0000","last_unstale":"2020-03-19T23:11:04.654160+0000","last_undegraded":"2020-03-19T23:11:04.654160+0000","last_fullsized":"2020-03-19T23:11:04.654160+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.648376+0000","last_change":"2020-03-19T23:11:01.250472+0000","last_active":"2020-03-19T23:11:04.648376+0000","last_peered":"2020-03-19T23:11:04.648376+0000","last_clean":"2020-03-19T23:11:04.648376+0000","last_became_active":"2020-03-19T23:11:01.250084+0000","last_became_peered":"2020-03-19T23:11:01.250084+0000","last_unstale":"2020-03-19T23:11:04.648376+0000","last_undegraded":"2020-03-19T23:11:04.648376+0000","last_fullsized":"2020-03-19T23:11:04.648376+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'3","reported_seq":"13","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.324272+0000","last_change":"2020-03-19T23:11:04.285615+0000","last_active":"2020-03-19T23:11:07.324272+0000","last_peered":"2020-03-19T23:11:07.324272+0000","last_clean":"2020-03-19T23:11:07.324272+0000","last_became_active":"2020-03-19T23:11:04.284829+0000","last_became_peered":"2020-03-19T23:11:04.284829+0000","last_unstale":"2020-03-19T23:11:07.324272+0000","last_undegraded":"2020-03-19T23:11:07.324272+0000","last_fullsized":"2020-03-19T23:11:07.324272+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.654566+0000","last_change":"2020-03-19T23:11:04.654105+0000","last_active":"2020-03-19T23:11:04.654566+0000","last_peered":"2020-03-19T23:11:04.654566+0000","last_clean":"2020-03-19T23:11:04.654566+0000","last_became_active":"2020-03-19T23:11:04.653741+0000","last_became_peered":"2020-03-19T23:11:04.653741+0000","last_unstale":"2020-03-19T23:11:04.654566+0000","last_undegraded":"2020-03-19T23:11:04.654566+0000","last_fullsized":"2020-03-19T23:11:04.654566+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.647891+0000","last_change":"2020-03-19T23:11:01.251743+0000","last_active":"2020-03-19T23:11:04.647891+0000","last_peered":"2020-03-19T23:11:04.647891+0000","last_clean":"2020-03-19T23:11:04.647891+0000","last_became_active":"2020-03-19T23:11:01.251378+0000","last_became_peered":"2020-03-19T23:11:01.251378+0000","last_unstale":"2020-03-19T23:11:04.647891+0000","last_undegraded":"2020-03-19T23:11:04.647891+0000","last_fullsized":"2020-03-19T23:11:04.647891+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"17","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.324921+0000","last_change":"2020-03-19T23:11:04.285389+0000","last_active":"2020-03-19T23:11:07.324921+0000","last_peered":"2020-03-19T23:11:07.324921+0000","last_clean":"2020-03-19T23:11:07.324921+0000","last_became_active":"2020-03-19T23:11:04.284979+0000","last_became_peered":"2020-03-19T23:11:04.284979+0000","last_unstale":"2020-03-19T23:11:07.324921+0000","last_undegraded":"2020-03-19T23:11:07.324921+0000","last_fullsized":"2020-03-19T23:11:07.324921+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653068+0000","last_change":"2020-03-19T23:11:04.650535+0000","last_active":"2020-03-19T23:11:04.653068+0000","last_peered":"2020-03-19T23:11:04.653068+0000","last_clean":"2020-03-19T23:11:04.653068+0000","last_became_active":"2020-03-19T23:11:04.649316+0000","last_became_peered":"2020-03-19T23:11:04.649316+0000","last_unstale":"2020-03-19T23:11:04.653068+0000","last_undegraded":"2020-03-19T23:11:04.653068+0000","last_fullsized":"2020-03-19T23:11:04.653068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.647577+0000","last_change":"2020-03-19T23:11:01.248950+0000","last_active":"2020-03-19T23:11:04.647577+0000","last_peered":"2020-03-19T23:11:04.647577+0000","last_clean":"2020-03-19T23:11:04.647577+0000","last_became_active":"2020-03-19T23:11:01.248513+0000","last_became_peered":"2020-03-19T23:11:01.248513+0000","last_unstale":"2020-03-19T23:11:04.647577+0000","last_undegraded":"2020-03-19T23:11:04.647577+0000","last_fullsized":"2020-03-19T23:11:04.647577+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.319991+0000","last_change":"2020-03-19T23:11:04.288163+0000","last_active":"2020-03-19T23:11:07.319991+0000","last_peered":"2020-03-19T23:11:07.319991+0000","last_clean":"2020-03-19T23:11:07.319991+0000","last_became_active":"2020-03-19T23:11:04.287139+0000","last_became_peered":"2020-03-19T23:11:04.287139+0000","last_unstale":"2020-03-19T23:11:07.319991+0000","last_undegraded":"2020-03-19T23:11:07.319991+0000","last_fullsized":"2020-03-19T23:11:07.319991+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.654283+0000","last_change":"2020-03-19T23:11:04.654111+0000","last_active":"2020-03-19T23:11:04.654283+0000","last_peered":"2020-03-19T23:11:04.654283+0000","last_clean":"2020-03-19T23:11:04.654283+0000","last_became_active":"2020-03-19T23:11:04.653768+0000","last_became_peered":"2020-03-19T23:11:04.653768+0000","last_unstale":"2020-03-19T23:11:04.654283+0000","last_undegraded":"2020-03-19T23:11:04.654283+0000","last_fullsized":"2020-03-19T23:11:04.654283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.645773+0000","last_change":"2020-03-19T23:11:01.252367+0000","last_active":"2020-03-19T23:11:04.645773+0000","last_peered":"2020-03-19T23:11:04.645773+0000","last_clean":"2020-03-19T23:11:04.645773+0000","last_became_active":"2020-03-19T23:11:01.251474+0000","last_became_peered":"2020-03-19T23:11:01.251474+0000","last_unstale":"2020-03-19T23:11:04.645773+0000","last_undegraded":"2020-03-19T23:11:04.645773+0000","last_fullsized":"2020-03-19T23:11:04.645773+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.648269+0000","last_change":"2020-03-19T23:11:04.287593+0000","last_active":"2020-03-19T23:11:04.648269+0000","last_peered":"2020-03-19T23:11:04.648269+0000","last_clean":"2020-03-19T23:11:04.648269+0000","last_became_active":"2020-03-19T23:11:04.286797+0000","last_became_peered":"2020-03-19T23:11:04.286797+0000","last_unstale":"2020-03-19T23:11:04.648269+0000","last_undegraded":"2020-03-19T23:11:04.648269+0000","last_fullsized":"2020-03-19T23:11:04.648269+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653128+0000","last_change":"2020-03-19T23:11:04.650882+0000","last_active":"2020-03-19T23:11:04.653128+0000","last_peered":"2020-03-19T23:11:04.653128+0000","last_clean":"2020-03-19T23:11:04.653128+0000","last_became_active":"2020-03-19T23:11:04.650546+0000","last_became_peered":"2020-03-19T23:11:04.650546+0000","last_unstale":"2020-03-19T23:11:04.653128+0000","last_undegraded":"2020-03-19T23:11:04.653128+0000","last_fullsized":"2020-03-19T23:11:04.653128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.649563+0000","last_change":"2020-03-19T23:11:01.249741+0000","last_active":"2020-03-19T23:11:04.649563+0000","last_peered":"2020-03-19T23:11:04.649563+0000","last_clean":"2020-03-19T23:11:04.649563+0000","last_became_active":"2020-03-19T23:11:01.249356+0000","last_became_peered":"2020-03-19T23:11:01.249356+0000","last_unstale":"2020-03-19T23:11:04.649563+0000","last_undegraded":"2020-03-19T23:11:04.649563+0000","last_fullsized":"2020-03-19T23:11:04.649563+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"15","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.324070+0000","last_change":"2020-03-19T23:11:04.284232+0000","last_active":"2020-03-19T23:11:07.324070+0000","last_peered":"2020-03-19T23:11:07.324070+0000","last_clean":"2020-03-19T23:11:07.324070+0000","last_became_active":"2020-03-19T23:11:04.283884+0000","last_became_peered":"2020-03-19T23:11:04.283884+0000","last_unstale":"2020-03-19T23:11:07.324070+0000","last_undegraded":"2020-03-19T23:11:07.324070+0000","last_fullsized":"2020-03-19T23:11:07.324070+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653060+0000","last_change":"2020-03-19T23:11:04.652133+0000","last_active":"2020-03-19T23:11:04.653060+0000","last_peered":"2020-03-19T23:11:04.653060+0000","last_clean":"2020-03-19T23:11:04.653060+0000","last_became_active":"2020-03-19T23:11:04.651761+0000","last_became_peered":"2020-03-19T23:11:04.651761+0000","last_unstale":"2020-03-19T23:11:04.653060+0000","last_undegraded":"2020-03-19T23:11:04.653060+0000","last_fullsized":"2020-03-19T23:11:04.653060+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"13","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.648086+0000","last_change":"2020-03-19T23:11:01.250680+0000","last_active":"2020-03-19T23:11:04.648086+0000","last_peered":"2020-03-19T23:11:04.648086+0000","last_clean":"2020-03-19T23:11:04.648086+0000","last_became_active":"2020-03-19T23:11:01.250171+0000","last_became_peered":"2020-03-19T23:11:01.250171+0000","last_unstale":"2020-03-19T23:11:04.648086+0000","last_undegraded":"2020-03-19T23:11:04.648086+0000","last_fullsized":"2020-03-19T23:11:04.648086+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"11","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:05.645359+0000","last_change":"2020-03-19T23:11:04.287858+0000","last_active":"2020-03-19T23:11:05.645359+0000","last_peered":"2020-03-19T23:11:05.645359+0000","last_clean":"2020-03-19T23:11:05.645359+0000","last_became_active":"2020-03-19T23:11:04.287457+0000","last_became_peered":"2020-03-19T23:11:04.287457+0000","last_unstale":"2020-03-19T23:11:05.645359+0000","last_undegraded":"2020-03-19T23:11:05.645359+0000","last_fullsized":"2020-03-19T23:11:05.645359+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.655287+0000","last_change":"2020-03-19T23:11:04.654429+0000","last_active":"2020-03-19T23:11:04.655287+0000","last_peered":"2020-03-19T23:11:04.655287+0000","last_clean":"2020-03-19T23:11:04.655287+0000","last_became_active":"2020-03-19T23:11:04.654078+0000","last_became_peered":"2020-03-19T23:11:04.654078+0000","last_unstale":"2020-03-19T23:11:04.655287+0000","last_undegraded":"2020-03-19T23:11:04.655287+0000","last_fullsized":"2020-03-19T23:11:04.655287+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.649254+0000","last_change":"2020-03-19T23:11:01.252623+0000","last_active":"2020-03-19T23:11:04.649254+0000","last_peered":"2020-03-19T23:11:04.649254+0000","last_clean":"2020-03-19T23:11:04.649254+0000","last_became_active":"2020-03-19T23:11:01.252205+0000","last_became_peered":"2020-03-19T23:11:01.252205+0000","last_unstale":"2020-03-19T23:11:04.649254+0000","last_undegraded":"2020-03-19T23:11:04.649254+0000","last_fullsized":"2020-03-19T23:11:04.649254+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"16","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.325527+0000","last_change":"2020-03-19T23:11:04.285188+0000","last_active":"2020-03-19T23:11:07.325527+0000","last_peered":"2020-03-19T23:11:07.325527+0000","last_clean":"2020-03-19T23:11:07.325527+0000","last_became_active":"2020-03-19T23:11:04.284579+0000","last_became_peered":"2020-03-19T23:11:04.284579+0000","last_unstale":"2020-03-19T23:11:07.325527+0000","last_undegraded":"2020-03-19T23:11:07.325527+0000","last_fullsized":"2020-03-19T23:11:07.325527+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.653327+0000","last_change":"2020-03-19T23:11:04.650468+0000","last_active":"2020-03-19T23:11:04.653327+0000","last_peered":"2020-03-19T23:11:04.653327+0000","last_clean":"2020-03-19T23:11:04.653327+0000","last_became_active":"2020-03-19T23:11:04.650124+0000","last_became_peered":"2020-03-19T23:11:04.650124+0000","last_unstale":"2020-03-19T23:11:04.653327+0000","last_undegraded":"2020-03-19T23:11:04.653327+0000","last_fullsized":"2020-03-19T23:11:04.653327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"12","reported_epoch":"15","state":"active+clean","last_fresh":"2020-03-19T23:11:04.647588+0000","last_change":"2020-03-19T23:11:01.249394+0000","last_active":"2020-03-19T23:11:04.647588+0000","last_peered":"2020-03-19T23:11:04.647588+0000","last_clean":"2020-03-19T23:11:04.647588+0000","last_became_active":"2020-03-19T23:11:01.249024+0000","last_became_peered":"2020-03-19T23:11:01.249024+0000","last_unstale":"2020-03-19T23:11:04.647588+0000","last_undegraded":"2020-03-19T23:11:04.647588+0000","last_fullsized":"2020-03-19T23:11:04.647588+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"16'8","reported_seq":"18","reported_epoch":"16","state":"active+clean","last_fresh":"2020-03-19T23:11:07.325123+0000","last_change":"2020-03-19T23:11:04.283517+0000","last_active":"2020-03-19T23:11:07.325123+0000","last_peered":"2020-03-19T23:11:07.325123+0000","last_clean":"2020-03-19T23:11:07.325123+0000","last_became_active":"2020-03-19T23:11:04.283065+0000","last_became_peered":"2020-03-19T23:11:04.283065+0000","last_unstale":"2020-03-19T23:11:07.325123+0000","last_undegraded":"2020-03-19T23:11:07.325123+0000","last_fullsized":"2020-03-19T23:11:07.325123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048724,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808876,"statfs":{"total":107374182400,"available":106300289024,"internally_reserved":1073741824,"allocated":143360,"data_stored":42553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048748,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808852,"statfs":{"total":107374182400,"available":106300264448,"internally_reserved":1073741824,"allocated":167936,"data_stored":44771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048752,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808848,"statfs":{"total":107374182400,"available":106300260352,"internally_reserved":1073741824,"allocated":172032,"data_stored":44805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]}]}} 2020-03-19T23:11:12.145 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-03-19T23:11:12.145 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-03-19T23:11:12.145 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-03-19T23:11:12.146 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-03-19T23:11:12.258 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82d033b700 1 Processor -- start 2020-03-19T23:11:12.258 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82d033b700 1 -- start start 2020-03-19T23:11:12.263 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82d033b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c8126dc0 0x7f82c81291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82d033b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c8062590 con 0x7f82c80667d0 2020-03-19T23:11:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82d033b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c80907d0 con 0x7f82c8126dc0 2020-03-19T23:11:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82cd8d6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c8126dc0 0x7f82c81291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82cd8d6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c8126dc0 0x7f82c81291b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60936/0 (socket says 172.21.15.38:60936) 2020-03-19T23:11:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82ce0d7700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f82c80667d0 0x7f82c8066bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44074/0 (socket says 172.21.15.38:44074) 2020-03-19T23:11:12.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.256+0000 7f82cd8d6700 1 -- 172.21.15.38:0/2945515561 learned_addr learned my addr 172.21.15.38:0/2945515561 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:12.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 -- 172.21.15.38:0/2945515561 >> v1:172.21.15.38:6789/0 conn(0x7f82c80667d0 legacy=0x7f82c8066bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 -- 172.21.15.38:0/2945515561 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c808d290 con 0x7f82c8126dc0 2020-03-19T23:11:12.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd0d5700 1 -- 172.21.15.38:0/2945515561 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720578356 0 0) 0x7f82c8062590 con 0x7f82c80667d0 2020-03-19T23:11:12.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/2945515561 >> v2:172.21.15.38:3300/0 conn(0x7f82c8126dc0 0x7f82c81291b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f82c4002f30 tx=0x7f82c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/2945515561 >> v2:172.21.15.38:3300/0 conn(0x7f82c8126dc0 0x7f82c81291b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f82c4002f30 tx=0x7f82c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/2945515561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8126dc0 0x7f82c81291b0 secure :-1 s=READY pgs=175 cs=0 l=1 rx=0x7f82c4002f30 tx=0x7f82c4007a40).ready entity=mon.0 client_cookie=56329c9750c4294f server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/2945515561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8126dc0 0x7f82c81291b0 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f82c4002f30 tx=0x7f82c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd0d5700 1 -- 172.21.15.38:0/2945515561 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f82c40114b0 con 0x7f82c8126dc0 2020-03-19T23:11:12.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/2945515561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8126dc0 0x7f82c81291b0 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f82c4002f30 tx=0x7f82c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd0d5700 1 -- 172.21.15.38:0/2945515561 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4011610 con 0x7f82c8126dc0 2020-03-19T23:11:12.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/2945515561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8126dc0 0x7f82c81291b0 secure :-1 s=THROTTLE_DONE pgs=175 cs=0 l=1 rx=0x7f82c4002f30 tx=0x7f82c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82d033b700 1 -- 172.21.15.38:0/2945515561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8126dc0 msgr2=0x7f82c81291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82cd0d5700 1 -- 172.21.15.38:0/2945515561 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f82c40118f0 con 0x7f82c8126dc0 2020-03-19T23:11:12.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.257+0000 7f82d033b700 1 --2- 172.21.15.38:0/2945515561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8126dc0 0x7f82c81291b0 secure :-1 s=READY pgs=175 cs=0 l=1 rx=0x7f82c4002f30 tx=0x7f82c4007a40).stop 2020-03-19T23:11:12.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.258+0000 7f82d033b700 1 -- 172.21.15.38:0/2945515561 shutdown_connections 2020-03-19T23:11:12.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.258+0000 7f82d033b700 1 --2- 172.21.15.38:0/2945515561 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c8126dc0 0x7f82c81291b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.258+0000 7f82d033b700 1 -- 172.21.15.38:0/2945515561 >> 172.21.15.38:0/2945515561 conn(0x7f82c8091540 msgr2=0x7f82c8091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.258+0000 7f82d033b700 1 -- 172.21.15.38:0/2945515561 shutdown_connections 2020-03-19T23:11:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.258+0000 7f82d033b700 1 -- 172.21.15.38:0/2945515561 wait complete. 2020-03-19T23:11:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.258+0000 7f82d033b700 1 Processor -- start 2020-03-19T23:11:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.258+0000 7f82d033b700 1 -- start start 2020-03-19T23:11:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82d033b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c80667d0 0x7f82c811a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:12.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82d033b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c811e310 con 0x7f82c8126dc0 2020-03-19T23:11:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82d033b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c811dee0 con 0x7f82c80667d0 2020-03-19T23:11:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82c80667d0 0x7f82c811a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82cd8d6700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f82c8126dc0 0x7f82c811b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44080/0 (socket says 172.21.15.38:44080) 2020-03-19T23:11:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82cd8d6700 1 -- 172.21.15.38:0/4174355664 learned_addr learned my addr 172.21.15.38:0/4174355664 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 -- 172.21.15.38:0/4174355664 >> v1:172.21.15.38:6789/0 conn(0x7f82c8126dc0 legacy=0x7f82c811b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145138303 0 0) 0x7f82c811e310 con 0x7f82c8126dc0 2020-03-19T23:11:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 -- 172.21.15.38:0/4174355664 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c811b350 con 0x7f82c80667d0 2020-03-19T23:11:12.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> v2:172.21.15.38:3300/0 conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> v2:172.21.15.38:3300/0 conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=READY pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f82bc011440 con 0x7f82c80667d0 2020-03-19T23:11:12.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.271 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc0115a0 con 0x7f82c80667d0 2020-03-19T23:11:12.271 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c811af60 con 0x7f82c80667d0 2020-03-19T23:11:12.271 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f82bc0118c0 con 0x7f82c80667d0 2020-03-19T23:11:12.271 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.259+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c811bd70 con 0x7f82c80667d0 2020-03-19T23:11:12.271 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.260+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.271 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.260+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f82bc00b0a0 con 0x7f82c80667d0 2020-03-19T23:11:12.271 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.260+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.272 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.260+0000 7f82bb7fe700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 0x7f82a802a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:12.272 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.260+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f82bc033fd0 con 0x7f82c80667d0 2020-03-19T23:11:12.272 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.260+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 0x7f82a802a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.272 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.261+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 0x7f82a802a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f82c4002ed0 tx=0x7f82c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.272 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.261+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 0x7f82a802a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f82c4002ed0 tx=0x7f82c4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.272 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.261+0000 7f82cd8d6700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 0x7f82a802a6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rx=0x7f82c4002ed0 tx=0x7f82c4007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.273 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.261+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c8056db0 con 0x7f82c80667d0 2020-03-19T23:11:12.273 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.264+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.273 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.264+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f82bc011af0 con 0x7f82c80667d0 2020-03-19T23:11:12.468 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.465+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f82c806aaf0 con 0x7f82c80667d0 2020-03-19T23:11:12.468 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.466+0000 7f82ce0d7700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=THROTTLE_DONE pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.468 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.466+0000 7f82bb7fe700 1 -- 172.21.15.38:0/4174355664 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f82bc007160 con 0x7f82c80667d0 2020-03-19T23:11:12.468 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 msgr2=0x7f82a802a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 0x7f82a802a6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rx=0x7f82c4002ed0 tx=0x7f82c4007a40).stop 2020-03-19T23:11:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 msgr2=0x7f82c811a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 secure :-1 s=READY pgs=177 cs=0 l=1 rx=0x7f82c811c380 tx=0x7f82bc0108f0).stop 2020-03-19T23:11:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 shutdown_connections 2020-03-19T23:11:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82c80667d0 0x7f82c811a500 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 --2- 172.21.15.38:0/4174355664 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f82a8028200 0x7f82a802a6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 >> 172.21.15.38:0/4174355664 conn(0x7f82c8091540 msgr2=0x7f82c8125860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:12.471 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 shutdown_connections 2020-03-19T23:11:12.471 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.467+0000 7f82d033b700 1 -- 172.21.15.38:0/4174355664 wait complete. 2020-03-19T23:11:12.479 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-03-19T23:11:12.479 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-03-19T23:11:12.480 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fba7cc700 1 Processor -- start 2020-03-19T23:11:12.593 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fba7cc700 1 -- start start 2020-03-19T23:11:12.598 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fba7cc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb4060a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fba7cc700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb408c000 con 0x7f4fb4066bd0 2020-03-19T23:11:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fba7cc700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb4061820 con 0x7f4fb408e450 2020-03-19T23:11:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fb3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb4060a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fb37fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4fb4066bd0 0x7f4fb4090450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44086/0 (socket says 172.21.15.38:44086) 2020-03-19T23:11:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.591+0000 7f4fb37fe700 1 -- 172.21.15.38:0/550714422 learned_addr learned my addr 172.21.15.38:0/550714422 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 -- 172.21.15.38:0/550714422 >> v1:172.21.15.38:6789/0 conn(0x7f4fb4066bd0 legacy=0x7f4fb4090450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 -- 172.21.15.38:0/550714422 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4067150 con 0x7f4fb408e450 2020-03-19T23:11:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb2ffd700 1 -- 172.21.15.38:0/550714422 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148360541 0 0) 0x7f4fb408c000 con 0x7f4fb4066bd0 2020-03-19T23:11:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/550714422 >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb4060a50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f4fa4002f30 tx=0x7f4fa4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/550714422 >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb4060a50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f4fa4002f30 tx=0x7f4fa4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/550714422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb4060a50 secure :-1 s=READY pgs=179 cs=0 l=1 rx=0x7f4fa4002f30 tx=0x7f4fa4007a40).ready entity=mon.0 client_cookie=8a931fc2d7ca510e server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/550714422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb4060a50 secure :-1 s=THROTTLE_DONE pgs=179 cs=0 l=1 rx=0x7f4fa4002f30 tx=0x7f4fa4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.601 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb2ffd700 1 -- 172.21.15.38:0/550714422 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fa40113f0 con 0x7f4fb408e450 2020-03-19T23:11:12.601 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/550714422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb4060a50 secure :-1 s=THROTTLE_DONE pgs=179 cs=0 l=1 rx=0x7f4fa4002f30 tx=0x7f4fa4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.601 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb2ffd700 1 -- 172.21.15.38:0/550714422 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4011550 con 0x7f4fb408e450 2020-03-19T23:11:12.601 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/550714422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb4060a50 secure :-1 s=THROTTLE_DONE pgs=179 cs=0 l=1 rx=0x7f4fa4002f30 tx=0x7f4fa4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.601 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fb2ffd700 1 -- 172.21.15.38:0/550714422 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fa4011830 con 0x7f4fb408e450 2020-03-19T23:11:12.601 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fba7cc700 1 -- 172.21.15.38:0/550714422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 msgr2=0x7f4fb4060a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.601 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fba7cc700 1 --2- 172.21.15.38:0/550714422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb4060a50 secure :-1 s=READY pgs=179 cs=0 l=1 rx=0x7f4fa4002f30 tx=0x7f4fa4007a40).stop 2020-03-19T23:11:12.602 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fba7cc700 1 -- 172.21.15.38:0/550714422 shutdown_connections 2020-03-19T23:11:12.602 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fba7cc700 1 --2- 172.21.15.38:0/550714422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb4060a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.602 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fba7cc700 1 -- 172.21.15.38:0/550714422 >> 172.21.15.38:0/550714422 conn(0x7f4fb4091540 msgr2=0x7f4fb4091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:12.602 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.592+0000 7f4fba7cc700 1 -- 172.21.15.38:0/550714422 shutdown_connections 2020-03-19T23:11:12.602 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.593+0000 7f4fba7cc700 1 -- 172.21.15.38:0/550714422 wait complete. 2020-03-19T23:11:12.602 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.593+0000 7f4fba7cc700 1 Processor -- start 2020-03-19T23:11:12.602 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.593+0000 7f4fba7cc700 1 -- start start 2020-03-19T23:11:12.603 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fba7cc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb411f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:12.603 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fba7cc700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb4121c40 con 0x7f4fb4066bd0 2020-03-19T23:11:12.603 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fba7cc700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb41214d0 con 0x7f4fb408e450 2020-03-19T23:11:12.603 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb3fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4fb4066bd0 0x7f4fb411ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44088/0 (socket says 172.21.15.38:44088) 2020-03-19T23:11:12.603 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb411f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.603 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb3fff700 1 -- 172.21.15.38:0/4163287178 learned_addr learned my addr 172.21.15.38:0/4163287178 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896636497 0 0) 0x7f4fb4121c40 con 0x7f4fb4066bd0 2020-03-19T23:11:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f94002fe0 con 0x7f4fb4066bd0 2020-03-19T23:11:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 -- 172.21.15.38:0/4163287178 >> v1:172.21.15.38:6789/0 conn(0x7f4fb4066bd0 legacy=0x7f4fb411ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 -- 172.21.15.38:0/4163287178 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4122930 con 0x7f4fb408e450 2020-03-19T23:11:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> v2:172.21.15.38:3300/0 conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.604 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=READY pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fa8005b20 con 0x7f4fb408e450 2020-03-19T23:11:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.594+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa8005c80 con 0x7f4fb408e450 2020-03-19T23:11:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb4122620 con 0x7f4fb408e450 2020-03-19T23:11:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4fa80060c0 con 0x7f4fb408e450 2020-03-19T23:11:12.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb4126780 con 0x7f4fb408e450 2020-03-19T23:11:12.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f4fa800a320 con 0x7f4fb408e450 2020-03-19T23:11:12.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.595+0000 7f4fb17fa700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 0x7f4f9402ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:12.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.596+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f4fa8034100 con 0x7f4fb408e450 2020-03-19T23:11:12.606 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.596+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 0x7f4f9402ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.596+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb4056db0 con 0x7f4fb408e450 2020-03-19T23:11:12.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.596+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 0x7f4f9402ac30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4fa4000bb0 tx=0x7f4fa4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.596+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 0x7f4f9402ac30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4fa4000bb0 tx=0x7f4fa4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.596+0000 7f4fb3fff700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 0x7f4f9402ac30 secure :-1 s=READY pgs=46 cs=0 l=1 rx=0x7f4fa4000bb0 tx=0x7f4fa4007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.599+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.599+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4fa8007700 con 0x7f4fb408e450 2020-03-19T23:11:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.750+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fb406aaf0 con 0x7f4fb408e450 2020-03-19T23:11:12.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.750+0000 7f4fb37fe700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=THROTTLE_DONE pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.750+0000 7f4fb17fa700 1 -- 172.21.15.38:0/4163287178 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8801 (secure 0 0 0) 0x7f4fa8011ce0 con 0x7f4fb408e450 2020-03-19T23:11:12.753 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:12.753 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:05.639978+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 msgr2=0x7f4f9402ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 0x7f4f9402ac30 secure :-1 s=READY pgs=46 cs=0 l=1 rx=0x7f4fa4000bb0 tx=0x7f4fa4007a40).stop 2020-03-19T23:11:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 msgr2=0x7f4fb411f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 secure :-1 s=READY pgs=181 cs=0 l=1 rx=0x7f4fb4121030 tx=0x7f4fa8010880).stop 2020-03-19T23:11:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 shutdown_connections 2020-03-19T23:11:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4f94028770 0x7f4f9402ac30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 --2- 172.21.15.38:0/4163287178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb408e450 0x7f4fb411f320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 >> 172.21.15.38:0/4163287178 conn(0x7f4fb4091540 msgr2=0x7f4fb40932e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 shutdown_connections 2020-03-19T23:11:12.755 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.752+0000 7f4fba7cc700 1 -- 172.21.15.38:0/4163287178 wait complete. 2020-03-19T23:11:12.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:10:59.540840+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'12', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:11:02.421156+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_metadata', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'metadata': u'cephfs'}}, u'options': {u'pg_autoscale_bias': 4, u'recovery_priority': 5, u'pg_num_min': 16}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'15', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:11:03.595719+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_data', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 3, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'data': u'cephfs'}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'16', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T23:11:12.765 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T23:11:12.880 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.878+0000 7f9839d5c700 1 Processor -- start 2020-03-19T23:11:12.881 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.878+0000 7f9839d5c700 1 -- start start 2020-03-19T23:11:12.885 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9839d5c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9834062280 0x7f9834123b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:12.886 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9839d5c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9834062660 con 0x7f9834123fa0 2020-03-19T23:11:12.886 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9839d5c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983408e940 con 0x7f9834062280 2020-03-19T23:11:12.886 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9838d5a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9834062280 0x7f9834123b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.886 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9838d5a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9834062280 0x7f9834123b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60954/0 (socket says 172.21.15.38:60954) 2020-03-19T23:11:12.886 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9833fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9834123fa0 0x7f983408f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44094/0 (socket says 172.21.15.38:44094) 2020-03-19T23:11:12.886 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9838d5a700 1 -- 172.21.15.38:0/2397835327 learned_addr learned my addr 172.21.15.38:0/2397835327 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:12.887 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9838d5a700 1 -- 172.21.15.38:0/2397835327 >> v1:172.21.15.38:6789/0 conn(0x7f9834123fa0 legacy=0x7f983408f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.887 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9838d5a700 1 -- 172.21.15.38:0/2397835327 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9834091fe0 con 0x7f9834062280 2020-03-19T23:11:12.887 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f98337fe700 1 -- 172.21.15.38:0/2397835327 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899172468 0 0) 0x7f98280010d0 con 0x7f9834123fa0 2020-03-19T23:11:12.887 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9838d5a700 1 --2- 172.21.15.38:0/2397835327 >> v2:172.21.15.38:3300/0 conn(0x7f9834062280 0x7f9834123b40 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f9824002f30 tx=0x7f9824007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.887 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.879+0000 7f9838d5a700 1 --2- 172.21.15.38:0/2397835327 >> v2:172.21.15.38:3300/0 conn(0x7f9834062280 0x7f9834123b40 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f9824002f30 tx=0x7f9824007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.887 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9838d5a700 1 --2- 172.21.15.38:0/2397835327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834062280 0x7f9834123b40 secure :-1 s=READY pgs=183 cs=0 l=1 rx=0x7f9824002f30 tx=0x7f9824007a40).ready entity=mon.0 client_cookie=753fe1ef95169e3e server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.887 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9838d5a700 1 --2- 172.21.15.38:0/2397835327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834062280 0x7f9834123b40 secure :-1 s=THROTTLE_DONE pgs=183 cs=0 l=1 rx=0x7f9824002f30 tx=0x7f9824007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.888 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f98337fe700 1 -- 172.21.15.38:0/2397835327 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f98240114b0 con 0x7f9834062280 2020-03-19T23:11:12.888 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9838d5a700 1 --2- 172.21.15.38:0/2397835327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834062280 0x7f9834123b40 secure :-1 s=THROTTLE_DONE pgs=183 cs=0 l=1 rx=0x7f9824002f30 tx=0x7f9824007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.888 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9838d5a700 1 --2- 172.21.15.38:0/2397835327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834062280 0x7f9834123b40 secure :-1 s=THROTTLE_DONE pgs=183 cs=0 l=1 rx=0x7f9824002f30 tx=0x7f9824007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.888 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f98337fe700 1 -- 172.21.15.38:0/2397835327 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9824011610 con 0x7f9834062280 2020-03-19T23:11:12.888 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f98337fe700 1 -- 172.21.15.38:0/2397835327 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f9824011930 con 0x7f9834062280 2020-03-19T23:11:12.888 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9839d5c700 1 -- 172.21.15.38:0/2397835327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834062280 msgr2=0x7f9834123b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.889 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9839d5c700 1 --2- 172.21.15.38:0/2397835327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834062280 0x7f9834123b40 secure :-1 s=READY pgs=183 cs=0 l=1 rx=0x7f9824002f30 tx=0x7f9824007a40).stop 2020-03-19T23:11:12.889 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9839d5c700 1 -- 172.21.15.38:0/2397835327 shutdown_connections 2020-03-19T23:11:12.889 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9839d5c700 1 --2- 172.21.15.38:0/2397835327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834062280 0x7f9834123b40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:12.889 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9839d5c700 1 -- 172.21.15.38:0/2397835327 >> 172.21.15.38:0/2397835327 conn(0x7f98340950e0 msgr2=0x7f9834091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:12.889 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9839d5c700 1 -- 172.21.15.38:0/2397835327 shutdown_connections 2020-03-19T23:11:12.889 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.880+0000 7f9839d5c700 1 -- 172.21.15.38:0/2397835327 wait complete. 2020-03-19T23:11:12.889 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.881+0000 7f9839d5c700 1 Processor -- start 2020-03-19T23:11:12.890 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.881+0000 7f9839d5c700 1 -- start start 2020-03-19T23:11:12.890 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.881+0000 7f9839d5c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9834123fa0 0x7f9834064d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:12.890 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.881+0000 7f9839d5c700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98340663a0 con 0x7f9834062280 2020-03-19T23:11:12.890 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.881+0000 7f9839d5c700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983410b3d0 con 0x7f9834123fa0 2020-03-19T23:11:12.890 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9838d5a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9834062280 0x7f9834064660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44096/0 (socket says 172.21.15.38:44096) 2020-03-19T23:11:12.890 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9834123fa0 0x7f9834064d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.890 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9838d5a700 1 -- 172.21.15.38:0/3356770345 learned_addr learned my addr 172.21.15.38:0/3356770345 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:12.891 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908699594 0 0) 0x7f98340663a0 con 0x7f9834062280 2020-03-19T23:11:12.891 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9814002fe0 con 0x7f9834062280 2020-03-19T23:11:12.891 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 -- 172.21.15.38:0/3356770345 >> v1:172.21.15.38:6789/0 conn(0x7f9834062280 legacy=0x7f9834064660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:12.891 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 -- 172.21.15.38:0/3356770345 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983410b7b0 con 0x7f9834123fa0 2020-03-19T23:11:12.891 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> v2:172.21.15.38:3300/0 conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.891 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> v2:172.21.15.38:3300/0 conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.891 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=READY pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f98280113f0 con 0x7f9834123fa0 2020-03-19T23:11:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828011550 con 0x7f9834123fa0 2020-03-19T23:11:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983410bab0 con 0x7f9834123fa0 2020-03-19T23:11:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f98280118b0 con 0x7f9834123fa0 2020-03-19T23:11:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.882+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9834066910 con 0x7f9834123fa0 2020-03-19T23:11:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f982800b0a0 con 0x7f9834123fa0 2020-03-19T23:11:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9831ffb700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 0x7f981402ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f9828033cb0 con 0x7f9834123fa0 2020-03-19T23:11:12.893 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983411dc60 con 0x7f9834123fa0 2020-03-19T23:11:12.894 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9838d5a700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 0x7f981402ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:12.894 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9838d5a700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 0x7f981402ac80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f9824000bb0 tx=0x7f9824007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.894 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9838d5a700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 0x7f981402ac80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f9824000bb0 tx=0x7f9824007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.894 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.883+0000 7f9838d5a700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 0x7f981402ac80 secure :-1 s=READY pgs=47 cs=0 l=1 rx=0x7f9824000bb0 tx=0x7f9824007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:12.895 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.887+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:12.895 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:12.887+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f9828005c00 con 0x7f9834123fa0 2020-03-19T23:11:13.037 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.035+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9834062660 con 0x7f9834123fa0 2020-03-19T23:11:13.037 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.035+0000 7f9833fff700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=THROTTLE_DONE pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.037 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.035+0000 7f9831ffb700 1 -- 172.21.15.38:0/3356770345 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9834062660 con 0x7f9834123fa0 2020-03-19T23:11:13.037 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:13.038 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 msgr2=0x7f981402ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.039 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 0x7f981402ac80 secure :-1 s=READY pgs=47 cs=0 l=1 rx=0x7f9824000bb0 tx=0x7f9824007a40).stop 2020-03-19T23:11:13.039 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 msgr2=0x7f9834064d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.039 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 secure :-1 s=READY pgs=185 cs=0 l=1 rx=0x7f9828000b80 tx=0x7f9828010860).stop 2020-03-19T23:11:13.039 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 shutdown_connections 2020-03-19T23:11:13.039 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f98140287c0 0x7f981402ac80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.039 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 --2- 172.21.15.38:0/3356770345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9834123fa0 0x7f9834064d70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.039 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 >> 172.21.15.38:0/3356770345 conn(0x7f98340950e0 msgr2=0x7f983408efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:13.040 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 shutdown_connections 2020-03-19T23:11:13.040 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.036+0000 7f9839d5c700 1 -- 172.21.15.38:0/3356770345 wait complete. 2020-03-19T23:11:13.048 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-03-19T23:11:13.164 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.162+0000 7fbcc3fff700 1 Processor -- start 2020-03-19T23:11:13.165 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.162+0000 7fbcc3fff700 1 -- start start 2020-03-19T23:11:13.169 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbcc4126830 0x7fbcc4128c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:13.170 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc3fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc4062660 con 0x7fbcc4062280 2020-03-19T23:11:13.170 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc3fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc4091fe0 con 0x7fbcc4126830 2020-03-19T23:11:13.170 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc27fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbcc4126830 0x7fbcc4128c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.170 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc2ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbcc4062280 0x7fbcc4123b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44102/0 (socket says 172.21.15.38:44102) 2020-03-19T23:11:13.170 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc27fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbcc4126830 0x7fbcc4128c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60962/0 (socket says 172.21.15.38:60962) 2020-03-19T23:11:13.170 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc2ffd700 1 -- 172.21.15.38:0/2369862588 learned_addr learned my addr 172.21.15.38:0/2369862588 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:13.171 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.163+0000 7fbcc27fc700 1 -- 172.21.15.38:0/2369862588 >> v1:172.21.15.38:6789/0 conn(0x7fbcc4062280 legacy=0x7fbcc4123b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.171 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc27fc700 1 -- 172.21.15.38:0/2369862588 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4123830 con 0x7fbcc4126830 2020-03-19T23:11:13.171 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc1ffb700 1 -- 172.21.15.38:0/2369862588 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496603028 0 0) 0x7fbcb00010d0 con 0x7fbcc4062280 2020-03-19T23:11:13.171 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/2369862588 >> v2:172.21.15.38:3300/0 conn(0x7fbcc4126830 0x7fbcc4128c20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fbcb4002f30 tx=0x7fbcb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.171 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/2369862588 >> v2:172.21.15.38:3300/0 conn(0x7fbcc4126830 0x7fbcc4128c20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fbcb4002f30 tx=0x7fbcb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.171 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/2369862588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4126830 0x7fbcc4128c20 secure :-1 s=READY pgs=187 cs=0 l=1 rx=0x7fbcb4002f30 tx=0x7fbcb4007a40).ready entity=mon.0 client_cookie=a5e88305f044a91b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.171 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/2369862588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4126830 0x7fbcc4128c20 secure :-1 s=THROTTLE_DONE pgs=187 cs=0 l=1 rx=0x7fbcb4002f30 tx=0x7fbcb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.172 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc1ffb700 1 -- 172.21.15.38:0/2369862588 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbcb40113f0 con 0x7fbcc4126830 2020-03-19T23:11:13.172 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/2369862588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4126830 0x7fbcc4128c20 secure :-1 s=THROTTLE_DONE pgs=187 cs=0 l=1 rx=0x7fbcb4002f30 tx=0x7fbcb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.172 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/2369862588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4126830 0x7fbcc4128c20 secure :-1 s=THROTTLE_DONE pgs=187 cs=0 l=1 rx=0x7fbcb4002f30 tx=0x7fbcb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.172 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc1ffb700 1 -- 172.21.15.38:0/2369862588 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4011550 con 0x7fbcc4126830 2020-03-19T23:11:13.172 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc1ffb700 1 -- 172.21.15.38:0/2369862588 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbcb4011830 con 0x7fbcc4126830 2020-03-19T23:11:13.172 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc3fff700 1 -- 172.21.15.38:0/2369862588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4126830 msgr2=0x7fbcc4128c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.172 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc3fff700 1 --2- 172.21.15.38:0/2369862588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4126830 0x7fbcc4128c20 secure :-1 s=READY pgs=187 cs=0 l=1 rx=0x7fbcb4002f30 tx=0x7fbcb4007a40).stop 2020-03-19T23:11:13.173 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc3fff700 1 -- 172.21.15.38:0/2369862588 shutdown_connections 2020-03-19T23:11:13.173 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc3fff700 1 --2- 172.21.15.38:0/2369862588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4126830 0x7fbcc4128c20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.173 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc3fff700 1 -- 172.21.15.38:0/2369862588 >> 172.21.15.38:0/2369862588 conn(0x7fbcc40950e0 msgr2=0x7fbcc4091180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:13.173 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc3fff700 1 -- 172.21.15.38:0/2369862588 shutdown_connections 2020-03-19T23:11:13.173 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.164+0000 7fbcc3fff700 1 -- 172.21.15.38:0/2369862588 wait complete. 2020-03-19T23:11:13.173 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.165+0000 7fbcc3fff700 1 Processor -- start 2020-03-19T23:11:13.173 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.165+0000 7fbcc3fff700 1 -- start start 2020-03-19T23:11:13.174 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbcc4062280 0x7fbcc411d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:13.174 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc3fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc4064470 con 0x7fbcc4126830 2020-03-19T23:11:13.174 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc3fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc411dc20 con 0x7fbcc4062280 2020-03-19T23:11:13.174 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbcc4062280 0x7fbcc411d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.174 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc27fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbcc4126830 0x7fbcc4062bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44106/0 (socket says 172.21.15.38:44106) 2020-03-19T23:11:13.174 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbcc4062280 0x7fbcc411d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60964/0 (socket says 172.21.15.38:60964) 2020-03-19T23:11:13.174 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc27fc700 1 -- 172.21.15.38:0/3881881771 learned_addr learned my addr 172.21.15.38:0/3881881771 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:13.175 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541407734 0 0) 0x7fbcc4064470 con 0x7fbcc4126830 2020-03-19T23:11:13.175 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 -- 172.21.15.38:0/3881881771 >> v1:172.21.15.38:6789/0 conn(0x7fbcc4126830 legacy=0x7fbcc4062bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.175 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 -- 172.21.15.38:0/3881881771 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4065930 con 0x7fbcc4062280 2020-03-19T23:11:13.175 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> v2:172.21.15.38:3300/0 conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.175 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> v2:172.21.15.38:3300/0 conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.175 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=READY pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.176 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.176 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbcb0005bc0 con 0x7fbcc4062280 2020-03-19T23:11:13.176 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.176 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.176 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0005d20 con 0x7fbcc4062280 2020-03-19T23:11:13.176 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc4063c60 con 0x7fbcc4062280 2020-03-19T23:11:13.176 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbcb00061f0 con 0x7fbcc4062280 2020-03-19T23:11:13.177 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.166+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc4065c50 con 0x7fbcc4062280 2020-03-19T23:11:13.177 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.177 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fbcb0006460 con 0x7fbcc4062280 2020-03-19T23:11:13.177 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.177 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcbbfff700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 0x7fbca002a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fbcb0033fb0 con 0x7fbcc4062280 2020-03-19T23:11:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 0x7fbca002a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc4062660 con 0x7fbcc4062280 2020-03-19T23:11:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 0x7fbca002a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbcb4010180 tx=0x7fbcb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 0x7fbca002a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbcb4010180 tx=0x7fbcb4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.167+0000 7fbcc27fc700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 0x7fbca002a6f0 secure :-1 s=READY pgs=48 cs=0 l=1 rx=0x7fbcb4010180 tx=0x7fbcb4007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.171+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.179 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.171+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fbcc4062660 con 0x7fbcc4062280 2020-03-19T23:11:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.318+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbcc4062660 con 0x7fbcc4062280 2020-03-19T23:11:13.321 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.319+0000 7fbcc2ffd700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=THROTTLE_DONE pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.321 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.319+0000 7fbcbbfff700 1 -- 172.21.15.38:0/3881881771 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbcc4062660 con 0x7fbcc4062280 2020-03-19T23:11:13.321 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:13.322 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 msgr2=0x7fbca002a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 0x7fbca002a6f0 secure :-1 s=READY pgs=48 cs=0 l=1 rx=0x7fbcb4010180 tx=0x7fbcb4007a40).stop 2020-03-19T23:11:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 msgr2=0x7fbcc411d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 secure :-1 s=READY pgs=189 cs=0 l=1 rx=0x7fbcb0000b80 tx=0x7fbcb0010910).stop 2020-03-19T23:11:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 shutdown_connections 2020-03-19T23:11:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbcc4062280 0x7fbcc411d7c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 --2- 172.21.15.38:0/3881881771 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbca0028230 0x7fbca002a6f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 >> 172.21.15.38:0/3881881771 conn(0x7fbcc40950e0 msgr2=0x7fbcc408efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:13.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 shutdown_connections 2020-03-19T23:11:13.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.320+0000 7fbcc3fff700 1 -- 172.21.15.38:0/3881881771 wait complete. 2020-03-19T23:11:13.333 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-03-19T23:11:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.447+0000 7f7e4e513700 1 Processor -- start 2020-03-19T23:11:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.447+0000 7f7e4e513700 1 -- start start 2020-03-19T23:11:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.447+0000 7f7e4e513700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e48127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.447+0000 7f7e4e513700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e481228a0 con 0x7f7e481224c0 2020-03-19T23:11:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.447+0000 7f7e4e513700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e48092200 con 0x7f7e48125840 2020-03-19T23:11:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e48127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e48127c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60972/0 (socket says 172.21.15.38:60972) 2020-03-19T23:11:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e47fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7e481224c0 0x7f7e48122ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44110/0 (socket says 172.21.15.38:44110) 2020-03-19T23:11:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 -- 172.21.15.38:0/445367212 learned_addr learned my addr 172.21.15.38:0/445367212 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 -- 172.21.15.38:0/445367212 >> v1:172.21.15.38:6789/0 conn(0x7f7e481224c0 legacy=0x7f7e48122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 -- 172.21.15.38:0/445367212 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48090b60 con 0x7f7e48125840 2020-03-19T23:11:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e46ffd700 1 -- 172.21.15.38:0/445367212 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321822399 0 0) 0x7f7e380010d0 con 0x7f7e481224c0 2020-03-19T23:11:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- 172.21.15.38:0/445367212 >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e48127c30 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- 172.21.15.38:0/445367212 >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e48127c30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- 172.21.15.38:0/445367212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e48127c30 secure :-1 s=READY pgs=191 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).ready entity=mon.0 client_cookie=3a58d9bc551b2523 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- 172.21.15.38:0/445367212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e48127c30 secure :-1 s=THROTTLE_DONE pgs=191 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e46ffd700 1 -- 172.21.15.38:0/445367212 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e3c0114b0 con 0x7f7e48125840 2020-03-19T23:11:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- 172.21.15.38:0/445367212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e48127c30 secure :-1 s=THROTTLE_DONE pgs=191 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e477fe700 1 --2- 172.21.15.38:0/445367212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e48127c30 secure :-1 s=THROTTLE_DONE pgs=191 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e46ffd700 1 -- 172.21.15.38:0/445367212 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c011610 con 0x7f7e48125840 2020-03-19T23:11:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e46ffd700 1 -- 172.21.15.38:0/445367212 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e3c011930 con 0x7f7e48125840 2020-03-19T23:11:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e4e513700 1 -- 172.21.15.38:0/445367212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 msgr2=0x7f7e48127c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.448+0000 7f7e4e513700 1 --2- 172.21.15.38:0/445367212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e48127c30 secure :-1 s=READY pgs=191 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).stop 2020-03-19T23:11:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.449+0000 7f7e4e513700 1 -- 172.21.15.38:0/445367212 shutdown_connections 2020-03-19T23:11:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.449+0000 7f7e4e513700 1 --2- 172.21.15.38:0/445367212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e48127c30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.449+0000 7f7e4e513700 1 -- 172.21.15.38:0/445367212 >> 172.21.15.38:0/445367212 conn(0x7f7e48091390 msgr2=0x7f7e48091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.449+0000 7f7e4e513700 1 -- 172.21.15.38:0/445367212 shutdown_connections 2020-03-19T23:11:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.449+0000 7f7e4e513700 1 -- 172.21.15.38:0/445367212 wait complete. 2020-03-19T23:11:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.449+0000 7f7e4e513700 1 Processor -- start 2020-03-19T23:11:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.449+0000 7f7e4e513700 1 -- start start 2020-03-19T23:11:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e4e513700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e4811fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e4e513700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4810d090 con 0x7f7e481224c0 2020-03-19T23:11:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e4e513700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4810df30 con 0x7f7e48125840 2020-03-19T23:11:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e47fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7e481224c0 0x7f7e4810aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44114/0 (socket says 172.21.15.38:44114) 2020-03-19T23:11:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e47fff700 1 -- 172.21.15.38:0/348559035 learned_addr learned my addr 172.21.15.38:0/348559035 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e4811fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e477fe700 1 -- 172.21.15.38:0/348559035 >> v1:172.21.15.38:6789/0 conn(0x7f7e481224c0 legacy=0x7f7e4810aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714902357 0 0) 0x7f7e4810d090 con 0x7f7e481224c0 2020-03-19T23:11:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e477fe700 1 -- 172.21.15.38:0/348559035 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48066060 con 0x7f7e48125840 2020-03-19T23:11:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> v2:172.21.15.38:3300/0 conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.450+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=READY pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e3c011990 con 0x7f7e48125840 2020-03-19T23:11:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c007150 con 0x7f7e48125840 2020-03-19T23:11:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e48066300 con 0x7f7e48125840 2020-03-19T23:11:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7e3c0073f0 con 0x7f7e48125840 2020-03-19T23:11:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4810c470 con 0x7f7e48125840 2020-03-19T23:11:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f7e3c0056c0 con 0x7f7e48125840 2020-03-19T23:11:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.451+0000 7f7e457fa700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 0x7f7e2802a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.452+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f7e3c034690 con 0x7f7e48125840 2020-03-19T23:11:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.452+0000 7f7e47fff700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 0x7f7e2802a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.463 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.452+0000 7f7e47fff700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 0x7f7e2802a770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7e38001180 tx=0x7f7e380017c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.463 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.452+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e48090b60 con 0x7f7e48125840 2020-03-19T23:11:13.463 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.452+0000 7f7e47fff700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 0x7f7e2802a770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7e38001180 tx=0x7f7e380017c0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.463 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.452+0000 7f7e47fff700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 0x7f7e2802a770 secure :-1 s=READY pgs=49 cs=0 l=1 rx=0x7f7e38001180 tx=0x7f7e380017c0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.464 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.455+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.464 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.455+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f7e48090b60 con 0x7f7e48125840 2020-03-19T23:11:13.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.603+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e48090b60 con 0x7f7e48125840 2020-03-19T23:11:13.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.603+0000 7f7e477fe700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=THROTTLE_DONE pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.605 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.603+0000 7f7e457fa700 1 -- 172.21.15.38:0/348559035 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7e48090b60 con 0x7f7e48125840 2020-03-19T23:11:13.605 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:13.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 msgr2=0x7f7e2802a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 0x7f7e2802a770 secure :-1 s=READY pgs=49 cs=0 l=1 rx=0x7f7e38001180 tx=0x7f7e380017c0).stop 2020-03-19T23:11:13.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 msgr2=0x7f7e4811fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 secure :-1 s=READY pgs=193 cs=0 l=1 rx=0x7f7e3c002f30 tx=0x7f7e3c007a40).stop 2020-03-19T23:11:13.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 shutdown_connections 2020-03-19T23:11:13.607 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7e280282b0 0x7f7e2802a770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 --2- 172.21.15.38:0/348559035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e48125840 0x7f7e4811fb70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 >> 172.21.15.38:0/348559035 conn(0x7f7e48091390 msgr2=0x7f7e481202f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:13.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 shutdown_connections 2020-03-19T23:11:13.608 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.604+0000 7f7e4e513700 1 -- 172.21.15.38:0/348559035 wait complete. 2020-03-19T23:11:13.617 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-03-19T23:11:13.730 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2278996700 1 Processor -- start 2020-03-19T23:11:13.730 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2278996700 1 -- start start 2020-03-19T23:11:13.738 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2278996700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2274128e40 0x7f227412b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:13.738 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2278996700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740661c0 con 0x7f2274092250 2020-03-19T23:11:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2278996700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227405b7b0 con 0x7f2274128e40 2020-03-19T23:11:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2271d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2274128e40 0x7f227412b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2271d9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2274128e40 0x7f227412b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60982/0 (socket says 172.21.15.38:60982) 2020-03-19T23:11:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f227259c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2274092250 0x7f2274126100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44120/0 (socket says 172.21.15.38:44120) 2020-03-19T23:11:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.728+0000 7f2271d9b700 1 -- 172.21.15.38:0/3789205158 learned_addr learned my addr 172.21.15.38:0/3789205158 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 -- 172.21.15.38:0/3789205158 >> v1:172.21.15.38:6789/0 conn(0x7f2274092250 legacy=0x7f2274126100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 -- 172.21.15.38:0/3789205158 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274066e20 con 0x7f2274128e40 2020-03-19T23:11:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 --2- 172.21.15.38:0/3789205158 >> v2:172.21.15.38:3300/0 conn(0x7f2274128e40 0x7f227412b230 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f2264002f30 tx=0x7f2264007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 --2- 172.21.15.38:0/3789205158 >> v2:172.21.15.38:3300/0 conn(0x7f2274128e40 0x7f227412b230 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f2264002f30 tx=0x7f2264007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 --2- 172.21.15.38:0/3789205158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274128e40 0x7f227412b230 secure :-1 s=READY pgs=195 cs=0 l=1 rx=0x7f2264002f30 tx=0x7f2264007a40).ready entity=mon.0 client_cookie=434cd6f14634c04b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 --2- 172.21.15.38:0/3789205158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274128e40 0x7f227412b230 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7f2264002f30 tx=0x7f2264007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f227159a700 1 -- 172.21.15.38:0/3789205158 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f22640114b0 con 0x7f2274128e40 2020-03-19T23:11:13.740 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 --2- 172.21.15.38:0/3789205158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274128e40 0x7f227412b230 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7f2264002f30 tx=0x7f2264007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2271d9b700 1 --2- 172.21.15.38:0/3789205158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274128e40 0x7f227412b230 secure :-1 s=THROTTLE_DONE pgs=195 cs=0 l=1 rx=0x7f2264002f30 tx=0x7f2264007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f227159a700 1 -- 172.21.15.38:0/3789205158 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2264011610 con 0x7f2274128e40 2020-03-19T23:11:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f227159a700 1 -- 172.21.15.38:0/3789205158 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2264011970 con 0x7f2274128e40 2020-03-19T23:11:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2278996700 1 -- 172.21.15.38:0/3789205158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274128e40 msgr2=0x7f227412b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2278996700 1 --2- 172.21.15.38:0/3789205158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274128e40 0x7f227412b230 secure :-1 s=READY pgs=195 cs=0 l=1 rx=0x7f2264002f30 tx=0x7f2264007a40).stop 2020-03-19T23:11:13.741 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2278996700 1 -- 172.21.15.38:0/3789205158 shutdown_connections 2020-03-19T23:11:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2278996700 1 --2- 172.21.15.38:0/3789205158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274128e40 0x7f227412b230 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2278996700 1 -- 172.21.15.38:0/3789205158 >> 172.21.15.38:0/3789205158 conn(0x7f2274092f60 msgr2=0x7f2274093370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2278996700 1 -- 172.21.15.38:0/3789205158 shutdown_connections 2020-03-19T23:11:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.729+0000 7f2278996700 1 -- 172.21.15.38:0/3789205158 wait complete. 2020-03-19T23:11:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.730+0000 7f2278996700 1 Processor -- start 2020-03-19T23:11:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.730+0000 7f2278996700 1 -- start start 2020-03-19T23:11:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f2278996700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2274092250 0x7f227410d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f2278996700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227410f130 con 0x7f2274128e40 2020-03-19T23:11:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f2278996700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227410eb00 con 0x7f2274092250 2020-03-19T23:11:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2274092250 0x7f227410d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f2271d9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2274128e40 0x7f227410d5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44126/0 (socket says 172.21.15.38:44126) 2020-03-19T23:11:13.743 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f2271d9b700 1 -- 172.21.15.38:0/4165242013 learned_addr learned my addr 172.21.15.38:0/4165242013 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:13.744 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 -- 172.21.15.38:0/4165242013 >> v1:172.21.15.38:6789/0 conn(0x7f2274128e40 legacy=0x7f227410d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.744 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430748802 0 0) 0x7f227410f130 con 0x7f2274128e40 2020-03-19T23:11:13.744 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 -- 172.21.15.38:0/4165242013 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22741236b0 con 0x7f2274092250 2020-03-19T23:11:13.744 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> v2:172.21.15.38:3300/0 conn(0x7f2274092250 0x7f227410d160 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.744 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> v2:172.21.15.38:3300/0 conn(0x7f2274092250 0x7f227410d160 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.744 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=READY pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.744 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f225c011430 con 0x7f2274092250 2020-03-19T23:11:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c011590 con 0x7f2274092250 2020-03-19T23:11:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f225c011830 con 0x7f2274092250 2020-03-19T23:11:13.745 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227410e310 con 0x7f2274092250 2020-03-19T23:11:13.746 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.731+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2274110870 con 0x7f2274092250 2020-03-19T23:11:13.746 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.746 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f225c00b0a0 con 0x7f2274092250 2020-03-19T23:11:13.746 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.746 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f226b7fe700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 0x7f225802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:13.746 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7f225c033f60 con 0x7f2274092250 2020-03-19T23:11:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f2271d9b700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 0x7f225802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f2271d9b700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 0x7f225802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2264010280 tx=0x7f2264007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f2271d9b700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 0x7f225802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2264010280 tx=0x7f2264007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.732+0000 7f2271d9b700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 0x7f225802a740 secure :-1 s=READY pgs=50 cs=0 l=1 rx=0x7f2264010280 tx=0x7f2264007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.733+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2274056db0 con 0x7f2274092250 2020-03-19T23:11:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.736+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.736+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f225c007160 con 0x7f2274092250 2020-03-19T23:11:13.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.934+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f227405e820 con 0x7f2274092250 2020-03-19T23:11:13.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.935+0000 7f227259c700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=THROTTLE_DONE pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:13.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.935+0000 7f226b7fe700 1 -- 172.21.15.38:0/4165242013 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1783 (secure 0 0 0) 0x7f225c007160 con 0x7f2274092250 2020-03-19T23:11:13.938 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:13.938 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-03-19T23:11:04.634508+0000","modified":"2020-03-19T23:11:08.305742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4202":{"gid":4202,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.38:6827/3900686918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3900686918},{"type":"v1","addr":"172.21.15.38:6827","nonce":3900686918}]},"join_fscid":-1,"export_targets":[],"features":4540138292836696063,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-03-19T23:11:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.936+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 msgr2=0x7f225802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.936+0000 7f2278996700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 0x7f225802a740 secure :-1 s=READY pgs=50 cs=0 l=1 rx=0x7f2264010280 tx=0x7f2264007a40).stop 2020-03-19T23:11:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.936+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 msgr2=0x7f227410d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.936+0000 7f2278996700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 secure :-1 s=READY pgs=197 cs=0 l=1 rx=0x7f2274092f00 tx=0x7f225c0108a0).stop 2020-03-19T23:11:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.937+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 shutdown_connections 2020-03-19T23:11:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.937+0000 7f2278996700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274092250 0x7f227410d160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.937+0000 7f2278996700 1 --2- 172.21.15.38:0/4165242013 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2258028280 0x7f225802a740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.937+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 >> 172.21.15.38:0/4165242013 conn(0x7f2274092f60 msgr2=0x7f2274128050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.937+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 shutdown_connections 2020-03-19T23:11:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:13.937+0000 7f2278996700 1 -- 172.21.15.38:0/4165242013 wait complete. 2020-03-19T23:11:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 6 2020-03-19T23:11:14.049 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {u'session_autoclose': 300, u'balancer': u'', u'up': {u'mds_0': 4202}, u'last_failure_osd_epoch': 0, u'in': [0], u'last_failure': 0, u'max_file_size': 1099511627776, u'explicitly_allowed_features': 0, u'damaged': [], u'tableserver': 0, u'metadata_pool': 2, u'failed': [], u'epoch': 6, u'flags': 18, u'max_mds': 1, u'compat': {u'compat': {}, u'ro_compat': {}, u'incompat': {u'feature_10': u'snaprealm v2', u'feature_8': u'no anchor table', u'feature_9': u'file layout v2', u'feature_2': u'client writeable ranges', u'feature_3': u'default file layouts on dirs', u'feature_1': u'base v0.20', u'feature_6': u'dirfrag is stored in omap', u'feature_4': u'dir inode in separate object', u'feature_5': u'mds uses versioned encoding'}}, u'min_compat_client': u'0 (unknown)', u'data_pools': [3], u'info': {u'gid_4202': {u'export_targets': [], u'name': u'a', u'incarnation': 5, u'state_seq': 2, u'state': u'up:active', u'gid': 4202, u'features': 4540138292836696063, u'rank': 0, u'flags': 0, u'join_fscid': -1, u'addrs': {u'addrvec': [{u'nonce': 3900686918, u'type': u'v2', u'addr': u'172.21.15.38:6826'}, {u'nonce': 3900686918, u'type': u'v1', u'addr': u'172.21.15.38:6827'}]}, u'addr': u'172.21.15.38:6827/3900686918'}}, u'fs_name': u'cephfs', u'created': u'2020-03-19T23:11:04.634508+0000', u'standby_count_wanted': 0, u'enabled': True, u'modified': u'2020-03-19T23:11:08.305742+0000', u'session_timeout': 60, u'stopped': [], u'ever_allowed_features': 0, u'root': 0} 2020-03-19T23:11:14.049 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-03-19T23:11:14.050 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2020-03-19T23:11:14.176 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2020-03-19T23:11:14.176 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3", 2020-03-19T23:11:14.176 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2020-03-19T23:11:14.177 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4202, 2020-03-19T23:11:14.177 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2020-03-19T23:11:14.177 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2020-03-19T23:11:14.177 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 6.8700616569999999, 2020-03-19T23:11:14.177 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 6, 2020-03-19T23:11:14.177 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 16, 2020-03-19T23:11:14.177 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 16, 2020-03-19T23:11:14.178 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 7.1588343940000003 2020-03-19T23:11:14.178 INFO:teuthology.orchestra.run.smithi038.stdout:} 2020-03-19T23:11:14.186 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3", "whoami": 0, "id": 4202, "want_state": "up:active", "state": "up:active", "rank_uptime": 6.8700616569999999, "mdsmap_epoch": 6, "osdmap_epoch": 16, "osdmap_epoch_barrier": 16, "uptime": 7.1588343940000003 } 2020-03-19T23:11:14.186 INFO:teuthology.run_tasks:Running task exec... 2020-03-19T23:11:14.197 INFO:teuthology.task.exec:Executing custom commands... 2020-03-19T23:11:14.197 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:11:14.197 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create data_cache 4' 2020-03-19T23:11:14.317 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.309+0000 7fc7ddcf6700 1 Processor -- start 2020-03-19T23:11:14.318 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.309+0000 7fc7ddcf6700 1 -- start start 2020-03-19T23:11:14.318 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7ddcf6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d808e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:14.318 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7ddcf6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d808ebc0 con 0x7fc7d808d7c0 2020-03-19T23:11:14.319 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7ddcf6700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d807eac0 con 0x7fc7d808e2b0 2020-03-19T23:11:14.319 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d808e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:14.319 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7d7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc7d808d7c0 0x7fc7d808dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44130/0 (socket says 172.21.15.38:44130) 2020-03-19T23:11:14.319 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d808e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60990/0 (socket says 172.21.15.38:60990) 2020-03-19T23:11:14.319 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7d7fff700 1 -- 172.21.15.38:0/2807189494 learned_addr learned my addr 172.21.15.38:0/2807189494 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 -- 172.21.15.38:0/2807189494 >> v1:172.21.15.38:6789/0 conn(0x7fc7d808d7c0 legacy=0x7fc7d808dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 -- 172.21.15.38:0/2807189494 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d8090bf0 con 0x7fc7d808e2b0 2020-03-19T23:11:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- 172.21.15.38:0/2807189494 >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d808e760 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fc7d807de70 tx=0x7fc7c00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- 172.21.15.38:0/2807189494 >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d808e760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fc7d807de70 tx=0x7fc7c00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- 172.21.15.38:0/2807189494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d808e760 secure :-1 s=READY pgs=199 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7c00088e0).ready entity=mon.0 client_cookie=2da6fdd0a627c1e5 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- 172.21.15.38:0/2807189494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d808e760 secure :-1 s=THROTTLE_DONE pgs=199 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7c00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7d77fe700 1 -- 172.21.15.38:0/2807189494 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc7c00113f0 con 0x7fc7d808e2b0 2020-03-19T23:11:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- 172.21.15.38:0/2807189494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d808e760 secure :-1 s=THROTTLE_DONE pgs=199 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7c00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7d77fe700 1 -- 172.21.15.38:0/2807189494 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0011550 con 0x7fc7d808e2b0 2020-03-19T23:11:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7cffff700 1 --2- 172.21.15.38:0/2807189494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d808e760 secure :-1 s=THROTTLE_DONE pgs=199 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7c00088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7d77fe700 1 -- 172.21.15.38:0/2807189494 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc7c0011830 con 0x7fc7d808e2b0 2020-03-19T23:11:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/2807189494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 msgr2=0x7fc7d808e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.310+0000 7fc7ddcf6700 1 --2- 172.21.15.38:0/2807189494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d808e760 secure :-1 s=READY pgs=199 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7c00088e0).stop 2020-03-19T23:11:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.311+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/2807189494 shutdown_connections 2020-03-19T23:11:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.311+0000 7fc7ddcf6700 1 --2- 172.21.15.38:0/2807189494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d808e760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.311+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/2807189494 >> 172.21.15.38:0/2807189494 conn(0x7fc7d8094790 msgr2=0x7fc7d8094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:14.325 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.311+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/2807189494 shutdown_connections 2020-03-19T23:11:14.325 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.311+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/2807189494 wait complete. 2020-03-19T23:11:14.325 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7ddcf6700 1 Processor -- start 2020-03-19T23:11:14.325 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7ddcf6700 1 -- start start 2020-03-19T23:11:14.325 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7ddcf6700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d8071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:14.325 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7ddcf6700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d8078280 con 0x7fc7d808d7c0 2020-03-19T23:11:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7ddcf6700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d8093280 con 0x7fc7d808e2b0 2020-03-19T23:11:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7cffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d8071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7cffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d8071160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60996/0 (socket says 172.21.15.38:60996) 2020-03-19T23:11:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7d7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc7d808d7c0 0x7fc7d8070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44134/0 (socket says 172.21.15.38:44134) 2020-03-19T23:11:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.312+0000 7fc7cffff700 1 -- 172.21.15.38:0/1330015280 learned_addr learned my addr 172.21.15.38:0/1330015280 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 -- 172.21.15.38:0/1330015280 >> v1:172.21.15.38:6789/0 conn(0x7fc7d808d7c0 legacy=0x7fc7d8070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 -- 172.21.15.38:0/1330015280 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d807d690 con 0x7fc7d808e2b0 2020-03-19T23:11:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> v2:172.21.15.38:3300/0 conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=READY pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7d57fa700 1 -- 172.21.15.38:0/1330015280 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc7c0007150 con 0x7fc7d808e2b0 2020-03-19T23:11:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7d57fa700 1 -- 172.21.15.38:0/1330015280 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c00072b0 con 0x7fc7d808e2b0 2020-03-19T23:11:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7d57fa700 1 -- 172.21.15.38:0/1330015280 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fc7c0007420 con 0x7fc7d808e2b0 2020-03-19T23:11:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d8072930 con 0x7fc7d808e2b0 2020-03-19T23:11:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d8076fe0 con 0x7fc7d808e2b0 2020-03-19T23:11:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7d57fa700 1 -- 172.21.15.38:0/1330015280 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fc7c00056c0 con 0x7fc7d808e2b0 2020-03-19T23:11:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7d57fa700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 0x7fc7bc02a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7d57fa700 1 -- 172.21.15.38:0/1330015280 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3403+0+0 (secure 0 0 0) 0x7fc7c0034960 con 0x7fc7d808e2b0 2020-03-19T23:11:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7d7fff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 0x7fc7bc02a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d8056db0 con 0x7fc7d808e2b0 2020-03-19T23:11:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7d7fff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 0x7fc7bc02a7d0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fc7d80781a0 tx=0x7fc7c8001c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7d7fff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 0x7fc7bc02a7d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fc7d80781a0 tx=0x7fc7c8001c00).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.314+0000 7fc7d7fff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 0x7fc7bc02a7d0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7fc7d80781a0 tx=0x7fc7c8001c00).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:14.330 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.317+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:14.330 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.317+0000 7fc7d57fa700 1 -- 172.21.15.38:0/1330015280 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fc7c0005950 con 0x7fc7d808e2b0 2020-03-19T23:11:14.467 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:14.465+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_cache", "pg_num": 4} v 0) v1 -- 0x7fc7d806ada0 con 0x7fc7d808e2b0 2020-03-19T23:11:15.315 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.313+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=READY pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.347 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.339+0000 7fc7cffff700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=THROTTLE_DONE pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.347 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.340+0000 7fc7d57fa700 1 -- 172.21.15.38:0/1330015280 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_cache", "pg_num": 4}]=0 pool 'data_cache' created v17) v1 ==== 123+0+0 (secure 0 0 0) 0x7fc7c0006070 con 0x7fc7d808e2b0 2020-03-19T23:11:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.342+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 msgr2=0x7fc7bc02a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.342+0000 7fc7ddcf6700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 0x7fc7bc02a7d0 secure :-1 s=READY pgs=51 cs=0 l=1 rx=0x7fc7d80781a0 tx=0x7fc7c8001c00).stop 2020-03-19T23:11:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.342+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 msgr2=0x7fc7d8071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.342+0000 7fc7ddcf6700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 secure :-1 s=READY pgs=201 cs=0 l=1 rx=0x7fc7d807de70 tx=0x7fc7d808edf0).stop 2020-03-19T23:11:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.343+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 shutdown_connections 2020-03-19T23:11:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.343+0000 7fc7ddcf6700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fc7bc028310 0x7fc7bc02a7d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:15.348 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.343+0000 7fc7ddcf6700 1 --2- 172.21.15.38:0/1330015280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d808e2b0 0x7fc7d8071160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.343+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 >> 172.21.15.38:0/1330015280 conn(0x7fc7d8094790 msgr2=0x7fc7d80637b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.344+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 shutdown_connections 2020-03-19T23:11:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.344+0000 7fc7ddcf6700 1 -- 172.21.15.38:0/1330015280 wait complete. 2020-03-19T23:11:15.349 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'data_cache' created 2020-03-19T23:11:15.359 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd tier add cephfs_data data_cache' 2020-03-19T23:11:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f4845d3a700 1 Processor -- start 2020-03-19T23:11:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f4845d3a700 1 -- start start 2020-03-19T23:11:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f4845d3a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f484008d7c0 0x7f484008dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f4845d3a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484008ebc0 con 0x7f484008e000 2020-03-19T23:11:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f4845d3a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484007eac0 con 0x7f484008d7c0 2020-03-19T23:11:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f483ffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f484008d7c0 0x7f484008dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f483ffff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f484008d7c0 0x7f484008dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32768/0 (socket says 172.21.15.38:32768) 2020-03-19T23:11:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f483f7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f484008e000 0x7f484008e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44140/0 (socket says 172.21.15.38:44140) 2020-03-19T23:11:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.476+0000 7f483ffff700 1 -- 172.21.15.38:0/3570000987 learned_addr learned my addr 172.21.15.38:0/3570000987 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 -- 172.21.15.38:0/3570000987 >> v1:172.21.15.38:6789/0 conn(0x7f484008e000 legacy=0x7f484008e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 -- 172.21.15.38:0/3570000987 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840090bf0 con 0x7f484008d7c0 2020-03-19T23:11:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 --2- 172.21.15.38:0/3570000987 >> v2:172.21.15.38:3300/0 conn(0x7f484008d7c0 0x7f484008dba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f484007de70 tx=0x7f48280088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.486 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 --2- 172.21.15.38:0/3570000987 >> v2:172.21.15.38:3300/0 conn(0x7f484008d7c0 0x7f484008dba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f484007de70 tx=0x7f48280088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.486 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 --2- 172.21.15.38:0/3570000987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008d7c0 0x7f484008dba0 secure :-1 s=READY pgs=203 cs=0 l=1 rx=0x7f484007de70 tx=0x7f48280088e0).ready entity=mon.0 client_cookie=f116179efe7cfa64 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:15.486 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 --2- 172.21.15.38:0/3570000987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008d7c0 0x7f484008dba0 secure :-1 s=THROTTLE_DONE pgs=203 cs=0 l=1 rx=0x7f484007de70 tx=0x7f48280088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.486 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483effd700 1 -- 172.21.15.38:0/3570000987 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48280114b0 con 0x7f484008d7c0 2020-03-19T23:11:15.486 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 --2- 172.21.15.38:0/3570000987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008d7c0 0x7f484008dba0 secure :-1 s=THROTTLE_DONE pgs=203 cs=0 l=1 rx=0x7f484007de70 tx=0x7f48280088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.486 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483effd700 1 -- 172.21.15.38:0/3570000987 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4828011610 con 0x7f484008d7c0 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483ffff700 1 --2- 172.21.15.38:0/3570000987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008d7c0 0x7f484008dba0 secure :-1 s=THROTTLE_DONE pgs=203 cs=0 l=1 rx=0x7f484007de70 tx=0x7f48280088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f4845d3a700 1 -- 172.21.15.38:0/3570000987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008d7c0 msgr2=0x7f484008dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f483effd700 1 -- 172.21.15.38:0/3570000987 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f48280118b0 con 0x7f484008d7c0 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f4845d3a700 1 --2- 172.21.15.38:0/3570000987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008d7c0 0x7f484008dba0 secure :-1 s=READY pgs=203 cs=0 l=1 rx=0x7f484007de70 tx=0x7f48280088e0).stop 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f4845d3a700 1 -- 172.21.15.38:0/3570000987 shutdown_connections 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f4845d3a700 1 --2- 172.21.15.38:0/3570000987 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008d7c0 0x7f484008dba0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.477+0000 7f4845d3a700 1 -- 172.21.15.38:0/3570000987 >> 172.21.15.38:0/3570000987 conn(0x7f4840094790 msgr2=0x7f4840094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:15.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.478+0000 7f4845d3a700 1 -- 172.21.15.38:0/3570000987 shutdown_connections 2020-03-19T23:11:15.488 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.478+0000 7f4845d3a700 1 -- 172.21.15.38:0/3570000987 wait complete. 2020-03-19T23:11:15.488 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.478+0000 7f4845d3a700 1 Processor -- start 2020-03-19T23:11:15.488 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f4845d3a700 1 -- start start 2020-03-19T23:11:15.488 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f4845d3a700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f484008e000 0x7f4840071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:15.488 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f4845d3a700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484007dcb0 con 0x7f484008d7c0 2020-03-19T23:11:15.488 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f4845d3a700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4840073ed0 con 0x7f484008e000 2020-03-19T23:11:15.488 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f483f7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f484008e000 0x7f4840071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:15.489 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f483ffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f484008d7c0 0x7f48400714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44142/0 (socket says 172.21.15.38:44142) 2020-03-19T23:11:15.489 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f483f7fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f484008e000 0x7f4840071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32772/0 (socket says 172.21.15.38:32772) 2020-03-19T23:11:15.489 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f483ffff700 1 -- 172.21.15.38:0/2091389341 learned_addr learned my addr 172.21.15.38:0/2091389341 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:15.489 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f483f7fe700 1 -- 172.21.15.38:0/2091389341 >> v1:172.21.15.38:6789/0 conn(0x7f484008d7c0 legacy=0x7f48400714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:15.489 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779356366 0 0) 0x7f484007dcb0 con 0x7f484008d7c0 2020-03-19T23:11:15.489 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.479+0000 7f483f7fe700 1 -- 172.21.15.38:0/2091389341 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840072fe0 con 0x7f484008e000 2020-03-19T23:11:15.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> v2:172.21.15.38:3300/0 conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> v2:172.21.15.38:3300/0 conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:15.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4830011410 con 0x7f484008e000 2020-03-19T23:11:15.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830011610 con 0x7f484008e000 2020-03-19T23:11:15.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4840073520 con 0x7f484008e000 2020-03-19T23:11:15.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4830011930 con 0x7f484008e000 2020-03-19T23:11:15.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4840074780 con 0x7f484008e000 2020-03-19T23:11:15.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f483000b0a0 con 0x7f484008e000 2020-03-19T23:11:15.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.480+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.481+0000 7f483cff9700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 0x7f482402a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:15.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.481+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3752+0+0 (secure 0 0 0) 0x7f4830033fe0 con 0x7f484008e000 2020-03-19T23:11:15.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.481+0000 7f483ffff700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 0x7f482402a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:15.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.481+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840073320 con 0x7f484008e000 2020-03-19T23:11:15.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.481+0000 7f483ffff700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 0x7f482402a720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f48280082e0 tx=0x7f484008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.481+0000 7f483ffff700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 0x7f482402a720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f48280082e0 tx=0x7f484008edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.481+0000 7f483ffff700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 0x7f482402a720 secure :-1 s=READY pgs=52 cs=0 l=1 rx=0x7f48280082e0 tx=0x7f484008edf0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:15.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.485+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:15.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.485+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4830005e20 con 0x7f484008e000 2020-03-19T23:11:15.655 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:15.653+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd tier add", "pool": "cephfs_data", "tierpool": "data_cache"} v 0) v1 -- 0x7f4840056db0 con 0x7f484008e000 2020-03-19T23:11:16.350 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.347+0000 7f483f7fe700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=THROTTLE_DONE pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.350 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.348+0000 7f483cff9700 1 -- 172.21.15.38:0/2091389341 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd tier add", "pool": "cephfs_data", "tierpool": "data_cache"}]=0 pool 'data_cache' is now (or already was) a tier of 'cephfs_data' v18) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4830007370 con 0x7f484008e000 2020-03-19T23:11:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 msgr2=0x7f482402a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 0x7f482402a720 secure :-1 s=READY pgs=52 cs=0 l=1 rx=0x7f48280082e0 tx=0x7f484008edf0).stop 2020-03-19T23:11:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 msgr2=0x7f4840071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:16.354 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rx=0x7f484008fdb0 tx=0x7f4830010880).stop 2020-03-19T23:11:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 shutdown_connections 2020-03-19T23:11:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4824028260 0x7f482402a720 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 --2- 172.21.15.38:0/2091389341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f484008e000 0x7f4840071bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.352+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 >> 172.21.15.38:0/2091389341 conn(0x7f4840094790 msgr2=0x7f48400637b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.353+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 shutdown_connections 2020-03-19T23:11:16.356 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.353+0000 7f4845d3a700 1 -- 172.21.15.38:0/2091389341 wait complete. 2020-03-19T23:11:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'data_cache' is now (or already was) a tier of 'cephfs_data' 2020-03-19T23:11:16.368 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd tier cache-mode data_cache writeback' 2020-03-19T23:11:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f71ec4700 1 Processor -- start 2020-03-19T23:11:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f71ec4700 1 -- start start 2020-03-19T23:11:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f71ec4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f71ec4700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c08ebc0 con 0x7f0f6c08e000 2020-03-19T23:11:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f71ec4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c07eac0 con 0x7f0f6c08d7c0 2020-03-19T23:11:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f6bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f6bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32776/0 (socket says 172.21.15.38:32776) 2020-03-19T23:11:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f63fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0f6c08e000 0x7f0f6c08e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44148/0 (socket says 172.21.15.38:44148) 2020-03-19T23:11:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.486+0000 7f0f6bfff700 1 -- 172.21.15.38:0/4267791264 learned_addr learned my addr 172.21.15.38:0/4267791264 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 -- 172.21.15.38:0/4267791264 >> v1:172.21.15.38:6789/0 conn(0x7f0f6c08e000 legacy=0x7f0f6c08e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 -- 172.21.15.38:0/4267791264 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c090bf0 con 0x7f0f6c08d7c0 2020-03-19T23:11:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6b7fe700 1 -- 172.21.15.38:0/4267791264 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060005677 0 0) 0x7f0f6c08ebc0 con 0x7f0f6c08e000 2020-03-19T23:11:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/4267791264 >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0f6c07de70 tx=0x7f0f5c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/4267791264 >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0f6c07de70 tx=0x7f0f5c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/4267791264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 secure :-1 s=READY pgs=207 cs=0 l=1 rx=0x7f0f6c07de70 tx=0x7f0f5c0088e0).ready entity=mon.0 client_cookie=75c3de73c5756a93 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/4267791264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 secure :-1 s=THROTTLE_DONE pgs=207 cs=0 l=1 rx=0x7f0f6c07de70 tx=0x7f0f5c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6b7fe700 1 -- 172.21.15.38:0/4267791264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0f5c0114b0 con 0x7f0f6c08d7c0 2020-03-19T23:11:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/4267791264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 secure :-1 s=THROTTLE_DONE pgs=207 cs=0 l=1 rx=0x7f0f6c07de70 tx=0x7f0f5c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6b7fe700 1 -- 172.21.15.38:0/4267791264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c011610 con 0x7f0f6c08d7c0 2020-03-19T23:11:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/4267791264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 secure :-1 s=THROTTLE_DONE pgs=207 cs=0 l=1 rx=0x7f0f6c07de70 tx=0x7f0f5c0088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f6b7fe700 1 -- 172.21.15.38:0/4267791264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0f5c0118b0 con 0x7f0f6c08d7c0 2020-03-19T23:11:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f71ec4700 1 -- 172.21.15.38:0/4267791264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08d7c0 msgr2=0x7f0f6c08dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f71ec4700 1 --2- 172.21.15.38:0/4267791264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 secure :-1 s=READY pgs=207 cs=0 l=1 rx=0x7f0f6c07de70 tx=0x7f0f5c0088e0).stop 2020-03-19T23:11:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f71ec4700 1 -- 172.21.15.38:0/4267791264 shutdown_connections 2020-03-19T23:11:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f71ec4700 1 --2- 172.21.15.38:0/4267791264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08d7c0 0x7f0f6c08dba0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.487+0000 7f0f71ec4700 1 -- 172.21.15.38:0/4267791264 >> 172.21.15.38:0/4267791264 conn(0x7f0f6c094790 msgr2=0x7f0f6c094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.488+0000 7f0f71ec4700 1 -- 172.21.15.38:0/4267791264 shutdown_connections 2020-03-19T23:11:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.488+0000 7f0f71ec4700 1 -- 172.21.15.38:0/4267791264 wait complete. 2020-03-19T23:11:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.488+0000 7f0f71ec4700 1 Processor -- start 2020-03-19T23:11:16.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f71ec4700 1 -- start start 2020-03-19T23:11:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f71ec4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08e000 0x7f0f6c074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f71ec4700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c073d10 con 0x7f0f6c08d7c0 2020-03-19T23:11:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f71ec4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0747d0 con 0x7f0f6c08e000 2020-03-19T23:11:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f63fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08e000 0x7f0f6c074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f6bfff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0f6c08d7c0 0x7f0f6c071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44152/0 (socket says 172.21.15.38:44152) 2020-03-19T23:11:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f6bfff700 1 -- 172.21.15.38:0/61427158 learned_addr learned my addr 172.21.15.38:0/61427158 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f63fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08e000 0x7f0f6c074370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32778/0 (socket says 172.21.15.38:32778) 2020-03-19T23:11:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f63fff700 1 -- 172.21.15.38:0/61427158 >> v1:172.21.15.38:6789/0 conn(0x7f0f6c08d7c0 legacy=0x7f0f6c071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f63fff700 1 -- 172.21.15.38:0/61427158 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c073360 con 0x7f0f6c08e000 2020-03-19T23:11:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242813036 0 0) 0x7f0f6c073d10 con 0x7f0f6c08d7c0 2020-03-19T23:11:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> v2:172.21.15.38:3300/0 conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.489+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=READY pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0f5400d3c0 con 0x7f0f6c08e000 2020-03-19T23:11:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5400d520 con 0x7f0f6c08e000 2020-03-19T23:11:16.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c07d8f0 con 0x7f0f6c08e000 2020-03-19T23:11:16.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0f540145a0 con 0x7f0f6c08e000 2020-03-19T23:11:16.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c0724e0 con 0x7f0f6c08e000 2020-03-19T23:11:16.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f0f540121f0 con 0x7f0f6c08e000 2020-03-19T23:11:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f697fa700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 0x7f0f5002a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.490+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3760+0+0 (secure 0 0 0) 0x7f0f540360a0 con 0x7f0f6c08e000 2020-03-19T23:11:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.491+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c073b70 con 0x7f0f6c08e000 2020-03-19T23:11:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.491+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 0x7f0f5002a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.491+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 0x7f0f5002a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0f5c010280 tx=0x7f0f6c08edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.491+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 0x7f0f5002a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0f5c010280 tx=0x7f0f6c08edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.491+0000 7f0f6bfff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 0x7f0f5002a6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f0f5c010280 tx=0x7f0f6c08edf0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.494+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.494+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f0f6c073b70 con 0x7f0f6c08e000 2020-03-19T23:11:16.668 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:16.666+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd tier cache-mode", "pool": "data_cache", "mode": "writeback"} v 0) v1 -- 0x7f0f6c073b70 con 0x7f0f6c08e000 2020-03-19T23:11:17.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.353+0000 7f0f63fff700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=THROTTLE_DONE pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.355 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.353+0000 7f0f697fa700 1 -- 172.21.15.38:0/61427158 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd tier cache-mode", "pool": "data_cache", "mode": "writeback"}]=0 set cache-mode for pool 'data_cache' to writeback v19) v1 ==== 159+0+0 (secure 0 0 0) 0x7f0f6c073b70 con 0x7f0f6c08e000 2020-03-19T23:11:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 msgr2=0x7f0f5002a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:17.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 0x7f0f5002a6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rx=0x7f0f5c010280 tx=0x7f0f6c08edf0).stop 2020-03-19T23:11:17.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 msgr2=0x7f0f6c074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:17.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 secure :-1 s=READY pgs=209 cs=0 l=1 rx=0x7f0f54000b80 tx=0x7f0f54001e40).stop 2020-03-19T23:11:17.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 shutdown_connections 2020-03-19T23:11:17.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0f50028230 0x7f0f5002a6f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:17.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 --2- 172.21.15.38:0/61427158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f6c08e000 0x7f0f6c074370 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:17.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 >> 172.21.15.38:0/61427158 conn(0x7f0f6c094790 msgr2=0x7f0f6c063990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:17.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 shutdown_connections 2020-03-19T23:11:17.359 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.355+0000 7f0f71ec4700 1 -- 172.21.15.38:0/61427158 wait complete. 2020-03-19T23:11:17.359 INFO:teuthology.orchestra.run.smithi038.stderr:set cache-mode for pool 'data_cache' to writeback 2020-03-19T23:11:17.370 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd tier set-overlay cephfs_data data_cache' 2020-03-19T23:11:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.488+0000 7fbf02f22700 1 Processor -- start 2020-03-19T23:11:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.488+0000 7fbf02f22700 1 -- start start 2020-03-19T23:11:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf02f22700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbefc08d7c0 0x7fbefc08dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf02f22700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbefc08ebc0 con 0x7fbefc08e000 2020-03-19T23:11:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf02f22700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbefc07eac0 con 0x7fbefc08d7c0 2020-03-19T23:11:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbefc08d7c0 0x7fbefc08dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbefc08d7c0 0x7fbefc08dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32784/0 (socket says 172.21.15.38:32784) 2020-03-19T23:11:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 -- 172.21.15.38:0/2199344825 learned_addr learned my addr 172.21.15.38:0/2199344825 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 -- 172.21.15.38:0/2199344825 >> v1:172.21.15.38:6789/0 conn(0x7fbefc08e000 legacy=0x7fbefc08e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbef3fff700 1 -- 172.21.15.38:0/2199344825 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316569358 0 0) 0x7fbefc08ebc0 con 0x7fbefc08e000 2020-03-19T23:11:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 -- 172.21.15.38:0/2199344825 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc090bf0 con 0x7fbefc08d7c0 2020-03-19T23:11:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- 172.21.15.38:0/2199344825 >> v2:172.21.15.38:3300/0 conn(0x7fbefc08d7c0 0x7fbefc08dba0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fbefc07de70 tx=0x7fbef80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- 172.21.15.38:0/2199344825 >> v2:172.21.15.38:3300/0 conn(0x7fbefc08d7c0 0x7fbefc08dba0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fbefc07de70 tx=0x7fbef80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- 172.21.15.38:0/2199344825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08d7c0 0x7fbefc08dba0 secure :-1 s=READY pgs=211 cs=0 l=1 rx=0x7fbefc07de70 tx=0x7fbef80088e0).ready entity=mon.0 client_cookie=9e60933eb3463b36 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- 172.21.15.38:0/2199344825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08d7c0 0x7fbefc08dba0 secure :-1 s=THROTTLE_DONE pgs=211 cs=0 l=1 rx=0x7fbefc07de70 tx=0x7fbef80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbef3fff700 1 -- 172.21.15.38:0/2199344825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbef80114b0 con 0x7fbefc08d7c0 2020-03-19T23:11:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- 172.21.15.38:0/2199344825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08d7c0 0x7fbefc08dba0 secure :-1 s=THROTTLE_DONE pgs=211 cs=0 l=1 rx=0x7fbefc07de70 tx=0x7fbef80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbef3fff700 1 -- 172.21.15.38:0/2199344825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8011610 con 0x7fbefc08d7c0 2020-03-19T23:11:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf014bf700 1 --2- 172.21.15.38:0/2199344825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08d7c0 0x7fbefc08dba0 secure :-1 s=THROTTLE_DONE pgs=211 cs=0 l=1 rx=0x7fbefc07de70 tx=0x7fbef80088e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbef3fff700 1 -- 172.21.15.38:0/2199344825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbef80118b0 con 0x7fbefc08d7c0 2020-03-19T23:11:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf02f22700 1 -- 172.21.15.38:0/2199344825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08d7c0 msgr2=0x7fbefc08dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.489+0000 7fbf02f22700 1 --2- 172.21.15.38:0/2199344825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08d7c0 0x7fbefc08dba0 secure :-1 s=READY pgs=211 cs=0 l=1 rx=0x7fbefc07de70 tx=0x7fbef80088e0).stop 2020-03-19T23:11:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.490+0000 7fbf02f22700 1 -- 172.21.15.38:0/2199344825 shutdown_connections 2020-03-19T23:11:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.490+0000 7fbf02f22700 1 --2- 172.21.15.38:0/2199344825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08d7c0 0x7fbefc08dba0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.490+0000 7fbf02f22700 1 -- 172.21.15.38:0/2199344825 >> 172.21.15.38:0/2199344825 conn(0x7fbefc094790 msgr2=0x7fbefc094ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.490+0000 7fbf02f22700 1 -- 172.21.15.38:0/2199344825 shutdown_connections 2020-03-19T23:11:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.490+0000 7fbf02f22700 1 -- 172.21.15.38:0/2199344825 wait complete. 2020-03-19T23:11:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.491+0000 7fbf02f22700 1 Processor -- start 2020-03-19T23:11:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.491+0000 7fbf02f22700 1 -- start start 2020-03-19T23:11:17.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.491+0000 7fbf02f22700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbefc08e000 0x7fbefc075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:17.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.491+0000 7fbf02f22700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbefc07d780 con 0x7fbefc08d7c0 2020-03-19T23:11:17.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.491+0000 7fbf02f22700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbefc092a90 con 0x7fbefc08e000 2020-03-19T23:11:17.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbefc08e000 0x7fbefc075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:17.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf014bf700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbefc08d7c0 0x7fbefc072a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44160/0 (socket says 172.21.15.38:44160) 2020-03-19T23:11:17.500 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf014bf700 1 -- 172.21.15.38:0/1194109900 learned_addr learned my addr 172.21.15.38:0/1194109900 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 -- 172.21.15.38:0/1194109900 >> v1:172.21.15.38:6789/0 conn(0x7fbefc08d7c0 legacy=0x7fbefc072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 -- 172.21.15.38:0/1194109900 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc07d9b0 con 0x7fbefc08e000 2020-03-19T23:11:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9833581 0 0) 0x7fbefc07d780 con 0x7fbefc08d7c0 2020-03-19T23:11:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> v2:172.21.15.38:3300/0 conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> v2:172.21.15.38:3300/0 conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=READY pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbeec001ea0 con 0x7fbefc08e000 2020-03-19T23:11:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00cb00 con 0x7fbefc08e000 2020-03-19T23:11:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fbeec00ce20 con 0x7fbefc08e000 2020-03-19T23:11:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc074b00 con 0x7fbefc08e000 2020-03-19T23:11:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.492+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc074150 con 0x7fbefc08e000 2020-03-19T23:11:17.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fbeec0064e0 con 0x7fbefc08e000 2020-03-19T23:11:17.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbef1ffb700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 0x7fbedc02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:17.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3760+0+0 (secure 0 0 0) 0x7fbeec033cc0 con 0x7fbefc08e000 2020-03-19T23:11:17.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbf014bf700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 0x7fbedc02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:17.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbefc056db0 con 0x7fbefc08e000 2020-03-19T23:11:17.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbf014bf700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 0x7fbedc02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fbef8010280 tx=0x7fbefc08edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbf014bf700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 0x7fbedc02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fbef8010280 tx=0x7fbefc08edf0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.493+0000 7fbf014bf700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 0x7fbedc02a740 secure :-1 s=READY pgs=54 cs=0 l=1 rx=0x7fbef8010280 tx=0x7fbefc08edf0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:17.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.497+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:17.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.497+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fbeec002de0 con 0x7fbefc08e000 2020-03-19T23:11:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:17.667+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd tier set-overlay", "pool": "cephfs_data", "overlaypool": "data_cache"} v 0) v1 -- 0x7fbefc06ada0 con 0x7fbefc08e000 2020-03-19T23:11:18.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.359+0000 7fbf00cbe700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=THROTTLE_DONE pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.359+0000 7fbef1ffb700 1 -- 172.21.15.38:0/1194109900 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd tier set-overlay", "pool": "cephfs_data", "overlaypool": "data_cache"}]=0 overlay for 'cephfs_data' is now (or already was) 'data_cache' v20) v1 ==== 182+0+0 (secure 0 0 0) 0x7fbeec0090f0 con 0x7fbefc08e000 2020-03-19T23:11:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 msgr2=0x7fbedc02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 0x7fbedc02a740 secure :-1 s=READY pgs=54 cs=0 l=1 rx=0x7fbef8010280 tx=0x7fbefc08edf0).stop 2020-03-19T23:11:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 msgr2=0x7fbefc075480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 secure :-1 s=READY pgs=213 cs=0 l=1 rx=0x7fbeec000b80 tx=0x7fbeec0013a0).stop 2020-03-19T23:11:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 shutdown_connections 2020-03-19T23:11:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fbedc028280 0x7fbedc02a740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 --2- 172.21.15.38:0/1194109900 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbefc08e000 0x7fbefc075480 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 >> 172.21.15.38:0/1194109900 conn(0x7fbefc094790 msgr2=0x7fbefc063990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 shutdown_connections 2020-03-19T23:11:18.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.362+0000 7fbf02f22700 1 -- 172.21.15.38:0/1194109900 wait complete. 2020-03-19T23:11:18.366 INFO:teuthology.orchestra.run.smithi038.stderr:overlay for 'cephfs_data' is now (or already was) 'data_cache' 2020-03-19T23:11:18.378 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache hit_set_type bloom' 2020-03-19T23:11:18.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.499+0000 7f25832b4700 1 Processor -- start 2020-03-19T23:11:18.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.499+0000 7f25832b4700 1 -- start start 2020-03-19T23:11:18.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f25832b4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c08b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f25832b4700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257c067990 con 0x7f257c0655d0 2020-03-19T23:11:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f25832b4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c067bc0 con 0x7f257c08b5e0 2020-03-19T23:11:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f2582ab3700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c08b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f25822b2700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f257c0655d0 0x7f257c065a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44166/0 (socket says 172.21.15.38:44166) 2020-03-19T23:11:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f2582ab3700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c08b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32794/0 (socket says 172.21.15.38:32794) 2020-03-19T23:11:18.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f25822b2700 1 -- 172.21.15.38:0/2370599572 learned_addr learned my addr 172.21.15.38:0/2370599572 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f2582ab3700 1 -- 172.21.15.38:0/2370599572 >> v1:172.21.15.38:6789/0 conn(0x7f257c0655d0 legacy=0x7f257c065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f2582ab3700 1 -- 172.21.15.38:0/2370599572 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c08d000 con 0x7f257c08b5e0 2020-03-19T23:11:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.500+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2370599572 >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c08b9e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f2578002f30 tx=0x7f2578007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2370599572 >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c08b9e0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f2578002f30 tx=0x7f2578007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2370599572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c08b9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rx=0x7f2578002f30 tx=0x7f2578007a40).ready entity=mon.0 client_cookie=2f6f5f0ab3d48c7d server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2370599572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c08b9e0 secure :-1 s=THROTTLE_DONE pgs=215 cs=0 l=1 rx=0x7f2578002f30 tx=0x7f2578007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2581ab1700 1 -- 172.21.15.38:0/2370599572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2578008e80 con 0x7f257c08b5e0 2020-03-19T23:11:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2370599572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c08b9e0 secure :-1 s=THROTTLE_DONE pgs=215 cs=0 l=1 rx=0x7f2578002f30 tx=0x7f2578007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2370599572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c08b9e0 secure :-1 s=THROTTLE_DONE pgs=215 cs=0 l=1 rx=0x7f2578002f30 tx=0x7f2578007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2581ab1700 1 -- 172.21.15.38:0/2370599572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25780113f0 con 0x7f257c08b5e0 2020-03-19T23:11:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f2581ab1700 1 -- 172.21.15.38:0/2370599572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f25780116d0 con 0x7f257c08b5e0 2020-03-19T23:11:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f25832b4700 1 -- 172.21.15.38:0/2370599572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 msgr2=0x7f257c08b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f25832b4700 1 --2- 172.21.15.38:0/2370599572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c08b9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rx=0x7f2578002f30 tx=0x7f2578007a40).stop 2020-03-19T23:11:18.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f25832b4700 1 -- 172.21.15.38:0/2370599572 shutdown_connections 2020-03-19T23:11:18.508 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f25832b4700 1 --2- 172.21.15.38:0/2370599572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c08b9e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:18.508 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f25832b4700 1 -- 172.21.15.38:0/2370599572 >> 172.21.15.38:0/2370599572 conn(0x7f257c025010 msgr2=0x7f257c025420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:18.508 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f25832b4700 1 -- 172.21.15.38:0/2370599572 shutdown_connections 2020-03-19T23:11:18.508 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.501+0000 7f25832b4700 1 -- 172.21.15.38:0/2370599572 wait complete. 2020-03-19T23:11:18.508 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.502+0000 7f25832b4700 1 Processor -- start 2020-03-19T23:11:18.508 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.502+0000 7f25832b4700 1 -- start start 2020-03-19T23:11:18.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25832b4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c088920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:18.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25832b4700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257c08aff0 con 0x7f257c0655d0 2020-03-19T23:11:18.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25832b4700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c096470 con 0x7f257c08b5e0 2020-03-19T23:11:18.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f2582ab3700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f257c0655d0 0x7f257c088210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44170/0 (socket says 172.21.15.38:44170) 2020-03-19T23:11:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c088920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f2582ab3700 1 -- 172.21.15.38:0/2425909001 learned_addr learned my addr 172.21.15.38:0/2425909001 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234316541 0 0) 0x7f257c08aff0 con 0x7f257c0655d0 2020-03-19T23:11:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2564003230 con 0x7f257c0655d0 2020-03-19T23:11:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 -- 172.21.15.38:0/2425909001 >> v1:172.21.15.38:6789/0 conn(0x7f257c0655d0 legacy=0x7f257c088210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 -- 172.21.15.38:0/2425909001 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c096680 con 0x7f257c08b5e0 2020-03-19T23:11:18.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> v2:172.21.15.38:3300/0 conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=READY pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f256c005b20 con 0x7f257c08b5e0 2020-03-19T23:11:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c005c80 con 0x7f257c08b5e0 2020-03-19T23:11:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c0899d0 con 0x7f257c08b5e0 2020-03-19T23:11:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f256c0060c0 con 0x7f257c08b5e0 2020-03-19T23:11:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.503+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c062de0 con 0x7f257c08b5e0 2020-03-19T23:11:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.504+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.504+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f256c00a320 con 0x7f257c08b5e0 2020-03-19T23:11:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.504+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.504+0000 7f25737fe700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 0x7f256402acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.504+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3760+0+0 (secure 0 0 0) 0x7f256c034090 con 0x7f257c08b5e0 2020-03-19T23:11:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.504+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 0x7f256402acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.505+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 0x7f256402acb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2578000b80 tx=0x7f2578007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.505+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 0x7f256402acb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2578000b80 tx=0x7f2578007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.505+0000 7f2582ab3700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 0x7f256402acb0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f2578000b80 tx=0x7f2578007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.505+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c0896b0 con 0x7f257c08b5e0 2020-03-19T23:11:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.508+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.508+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f256c006220 con 0x7f257c08b5e0 2020-03-19T23:11:18.659 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:18.658+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_type", "val": "bloom"} v 0) v1 -- 0x7f257c056db0 con 0x7f257c08b5e0 2020-03-19T23:11:19.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.362+0000 7f25822b2700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=THROTTLE_DONE pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.362+0000 7f25737fe700 1 -- 172.21.15.38:0/2425909001 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_type", "val": "bloom"}]=0 set pool 4 hit_set_type to bloom v21) v1 ==== 153+0+0 (secure 0 0 0) 0x7f256c011c50 con 0x7f257c08b5e0 2020-03-19T23:11:19.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 msgr2=0x7f256402acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:19.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 0x7f256402acb0 secure :-1 s=READY pgs=55 cs=0 l=1 rx=0x7f2578000b80 tx=0x7f2578007a40).stop 2020-03-19T23:11:19.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 msgr2=0x7f257c088920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:19.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 secure :-1 s=READY pgs=217 cs=0 l=1 rx=0x7f257c063340 tx=0x7f256c0107f0).stop 2020-03-19T23:11:19.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 shutdown_connections 2020-03-19T23:11:19.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f25640287f0 0x7f256402acb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:19.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 --2- 172.21.15.38:0/2425909001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f257c08b5e0 0x7f257c088920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:19.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 >> 172.21.15.38:0/2425909001 conn(0x7f257c025010 msgr2=0x7f257c082390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:19.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 shutdown_connections 2020-03-19T23:11:19.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.364+0000 7f25832b4700 1 -- 172.21.15.38:0/2425909001 wait complete. 2020-03-19T23:11:19.368 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 hit_set_type to bloom 2020-03-19T23:11:19.379 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache hit_set_count 8' 2020-03-19T23:11:19.503 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f2279141700 1 Processor -- start 2020-03-19T23:11:19.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f2279141700 1 -- start start 2020-03-19T23:11:19.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f2279141700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f227408f930 0x7f227408fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:19.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f2279141700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227405c700 con 0x7f227407e780 2020-03-19T23:11:19.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f2279141700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2274090520 con 0x7f227408f930 2020-03-19T23:11:19.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f2272d9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f227408f930 0x7f227408fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:19.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f2272d9d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f227408f930 0x7f227408fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32804/0 (socket says 172.21.15.38:32804) 2020-03-19T23:11:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 -- 172.21.15.38:0/729581176 learned_addr learned my addr 172.21.15.38:0/729581176 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.499+0000 7f227359e700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f227407e780 0x7f227408f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44176/0 (socket says 172.21.15.38:44176) 2020-03-19T23:11:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 -- 172.21.15.38:0/729581176 >> v1:172.21.15.38:6789/0 conn(0x7f227407e780 legacy=0x7f227408f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 -- 172.21.15.38:0/729581176 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227407eb80 con 0x7f227408f930 2020-03-19T23:11:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 --2- 172.21.15.38:0/729581176 >> v2:172.21.15.38:3300/0 conn(0x7f227408f930 0x7f227408fdb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f2268002f30 tx=0x7f2268007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 --2- 172.21.15.38:0/729581176 >> v2:172.21.15.38:3300/0 conn(0x7f227408f930 0x7f227408fdb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f2268002f30 tx=0x7f2268007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 --2- 172.21.15.38:0/729581176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227408f930 0x7f227408fdb0 secure :-1 s=READY pgs=219 cs=0 l=1 rx=0x7f2268002f30 tx=0x7f2268007a40).ready entity=mon.0 client_cookie=57c64b0857b86a97 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 --2- 172.21.15.38:0/729581176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227408f930 0x7f227408fdb0 secure :-1 s=THROTTLE_DONE pgs=219 cs=0 l=1 rx=0x7f2268002f30 tx=0x7f2268007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f227259c700 1 -- 172.21.15.38:0/729581176 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f22680114b0 con 0x7f227408f930 2020-03-19T23:11:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 --2- 172.21.15.38:0/729581176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227408f930 0x7f227408fdb0 secure :-1 s=THROTTLE_DONE pgs=219 cs=0 l=1 rx=0x7f2268002f30 tx=0x7f2268007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2272d9d700 1 --2- 172.21.15.38:0/729581176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227408f930 0x7f227408fdb0 secure :-1 s=THROTTLE_DONE pgs=219 cs=0 l=1 rx=0x7f2268002f30 tx=0x7f2268007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f227259c700 1 -- 172.21.15.38:0/729581176 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268011610 con 0x7f227408f930 2020-03-19T23:11:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f227259c700 1 -- 172.21.15.38:0/729581176 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2268011970 con 0x7f227408f930 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2279141700 1 -- 172.21.15.38:0/729581176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227408f930 msgr2=0x7f227408fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.500+0000 7f2279141700 1 --2- 172.21.15.38:0/729581176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227408f930 0x7f227408fdb0 secure :-1 s=READY pgs=219 cs=0 l=1 rx=0x7f2268002f30 tx=0x7f2268007a40).stop 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.501+0000 7f2279141700 1 -- 172.21.15.38:0/729581176 shutdown_connections 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.501+0000 7f2279141700 1 --2- 172.21.15.38:0/729581176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227408f930 0x7f227408fdb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.501+0000 7f2279141700 1 -- 172.21.15.38:0/729581176 >> 172.21.15.38:0/729581176 conn(0x7f2274095460 msgr2=0x7f2274091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.501+0000 7f2279141700 1 -- 172.21.15.38:0/729581176 shutdown_connections 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.501+0000 7f2279141700 1 -- 172.21.15.38:0/729581176 wait complete. 2020-03-19T23:11:19.507 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f2279141700 1 Processor -- start 2020-03-19T23:11:19.508 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f2279141700 1 -- start start 2020-03-19T23:11:19.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f2279141700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f227407e780 0x7f2274070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:19.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f2279141700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2274073010 con 0x7f227408f930 2020-03-19T23:11:19.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f2279141700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227407dce0 con 0x7f227407e780 2020-03-19T23:11:19.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f227359e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f227407e780 0x7f2274070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:19.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f2272d9d700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f227408f930 0x7f2274071510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44180/0 (socket says 172.21.15.38:44180) 2020-03-19T23:11:19.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f227359e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f227407e780 0x7f2274070ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32806/0 (socket says 172.21.15.38:32806) 2020-03-19T23:11:19.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.502+0000 7f2272d9d700 1 -- 172.21.15.38:0/2743988568 learned_addr learned my addr 172.21.15.38:0/2743988568 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:19.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 -- 172.21.15.38:0/2743988568 >> v1:172.21.15.38:6789/0 conn(0x7f227408f930 legacy=0x7f2274071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:19.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735512906 0 0) 0x7f2274073010 con 0x7f227408f930 2020-03-19T23:11:19.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 -- 172.21.15.38:0/2743988568 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740746e0 con 0x7f227407e780 2020-03-19T23:11:19.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> v2:172.21.15.38:3300/0 conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> v2:172.21.15.38:3300/0 conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:19.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2264011410 con 0x7f227407e780 2020-03-19T23:11:19.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2264011570 con 0x7f227407e780 2020-03-19T23:11:19.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2274073e90 con 0x7f227407e780 2020-03-19T23:11:19.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2264011890 con 0x7f227407e780 2020-03-19T23:11:19.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2274073b10 con 0x7f227407e780 2020-03-19T23:11:19.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.503+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f226400b0a0 con 0x7f227407e780 2020-03-19T23:11:19.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f225bfff700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 0x7f225402a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:19.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f2264033f90 con 0x7f227407e780 2020-03-19T23:11:19.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f2272d9d700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 0x7f225402a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:19.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2274073400 con 0x7f227407e780 2020-03-19T23:11:19.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f2272d9d700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 0x7f225402a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2268010280 tx=0x7f2268007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f2272d9d700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 0x7f225402a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2268010280 tx=0x7f2268007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.504+0000 7f2272d9d700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 0x7f225402a710 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7f2268010280 tx=0x7f2268007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:19.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.507+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:19.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.507+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f2274073400 con 0x7f227407e780 2020-03-19T23:11:19.659 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:19.657+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_count", "val": "8"} v 0) v1 -- 0x7f2274073400 con 0x7f227407e780 2020-03-19T23:11:20.374 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.369+0000 7f227359e700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=THROTTLE_DONE pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.502 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.369+0000 7f225bfff700 1 -- 172.21.15.38:0/2743988568 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_count", "val": "8"}]=0 set pool 4 hit_set_count to 8 v22) v1 ==== 147+0+0 (secure 0 0 0) 0x7f2274073400 con 0x7f227407e780 2020-03-19T23:11:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 msgr2=0x7f225402a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 0x7f225402a710 secure :-1 s=READY pgs=56 cs=0 l=1 rx=0x7f2268010280 tx=0x7f2268007a40).stop 2020-03-19T23:11:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 msgr2=0x7f2274070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rx=0x7f2274094710 tx=0x7f2264010880).stop 2020-03-19T23:11:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 shutdown_connections 2020-03-19T23:11:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f227407e780 0x7f2274070ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 --2- 172.21.15.38:0/2743988568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f2254028250 0x7f225402a710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 >> 172.21.15.38:0/2743988568 conn(0x7f2274095460 msgr2=0x7f22740635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 shutdown_connections 2020-03-19T23:11:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.372+0000 7f2279141700 1 -- 172.21.15.38:0/2743988568 wait complete. 2020-03-19T23:11:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 hit_set_count to 8 2020-03-19T23:11:20.506 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache hit_set_period 3600' 2020-03-19T23:11:20.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.632+0000 7fd89bd21700 1 Processor -- start 2020-03-19T23:11:20.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.632+0000 7fd89bd21700 1 -- start start 2020-03-19T23:11:20.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd89bd21700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd89408f930 0x7fd89408fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:20.638 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd89bd21700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89405c700 con 0x7fd89407e780 2020-03-19T23:11:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd89bd21700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd894090520 con 0x7fd89408f930 2020-03-19T23:11:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd899abd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd89408f930 0x7fd89408fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd89a2be700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd89407e780 0x7fd89408f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44184/0 (socket says 172.21.15.38:44184) 2020-03-19T23:11:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd899abd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd89408f930 0x7fd89408fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32814/0 (socket says 172.21.15.38:32814) 2020-03-19T23:11:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd89a2be700 1 -- 172.21.15.38:0/3100845186 learned_addr learned my addr 172.21.15.38:0/3100845186 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd899abd700 1 -- 172.21.15.38:0/3100845186 >> v1:172.21.15.38:6789/0 conn(0x7fd89407e780 legacy=0x7fd89408f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd899abd700 1 -- 172.21.15.38:0/3100845186 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89407eb80 con 0x7fd89408f930 2020-03-19T23:11:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd8992bc700 1 -- 172.21.15.38:0/3100845186 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092440268 0 0) 0x7fd89405c700 con 0x7fd89407e780 2020-03-19T23:11:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.633+0000 7fd899abd700 1 --2- 172.21.15.38:0/3100845186 >> v2:172.21.15.38:3300/0 conn(0x7fd89408f930 0x7fd89408fdb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fd884002f30 tx=0x7fd884007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd899abd700 1 --2- 172.21.15.38:0/3100845186 >> v2:172.21.15.38:3300/0 conn(0x7fd89408f930 0x7fd89408fdb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fd884002f30 tx=0x7fd884007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd899abd700 1 --2- 172.21.15.38:0/3100845186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89408f930 0x7fd89408fdb0 secure :-1 s=READY pgs=223 cs=0 l=1 rx=0x7fd884002f30 tx=0x7fd884007a40).ready entity=mon.0 client_cookie=18e8d5a5d8a20f30 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd899abd700 1 --2- 172.21.15.38:0/3100845186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89408f930 0x7fd89408fdb0 secure :-1 s=THROTTLE_DONE pgs=223 cs=0 l=1 rx=0x7fd884002f30 tx=0x7fd884007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd8992bc700 1 -- 172.21.15.38:0/3100845186 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd8840113f0 con 0x7fd89408f930 2020-03-19T23:11:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd899abd700 1 --2- 172.21.15.38:0/3100845186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89408f930 0x7fd89408fdb0 secure :-1 s=THROTTLE_DONE pgs=223 cs=0 l=1 rx=0x7fd884002f30 tx=0x7fd884007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd8992bc700 1 -- 172.21.15.38:0/3100845186 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd884011550 con 0x7fd89408f930 2020-03-19T23:11:20.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd899abd700 1 --2- 172.21.15.38:0/3100845186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89408f930 0x7fd89408fdb0 secure :-1 s=THROTTLE_DONE pgs=223 cs=0 l=1 rx=0x7fd884002f30 tx=0x7fd884007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd8992bc700 1 -- 172.21.15.38:0/3100845186 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd884011830 con 0x7fd89408f930 2020-03-19T23:11:20.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd89bd21700 1 -- 172.21.15.38:0/3100845186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89408f930 msgr2=0x7fd89408fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:20.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd89bd21700 1 --2- 172.21.15.38:0/3100845186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89408f930 0x7fd89408fdb0 secure :-1 s=READY pgs=223 cs=0 l=1 rx=0x7fd884002f30 tx=0x7fd884007a40).stop 2020-03-19T23:11:20.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd89bd21700 1 -- 172.21.15.38:0/3100845186 shutdown_connections 2020-03-19T23:11:20.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd89bd21700 1 --2- 172.21.15.38:0/3100845186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89408f930 0x7fd89408fdb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:20.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd89bd21700 1 -- 172.21.15.38:0/3100845186 >> 172.21.15.38:0/3100845186 conn(0x7fd894095460 msgr2=0x7fd894091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:20.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd89bd21700 1 -- 172.21.15.38:0/3100845186 shutdown_connections 2020-03-19T23:11:20.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.634+0000 7fd89bd21700 1 -- 172.21.15.38:0/3100845186 wait complete. 2020-03-19T23:11:20.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.635+0000 7fd89bd21700 1 Processor -- start 2020-03-19T23:11:20.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.635+0000 7fd89bd21700 1 -- start start 2020-03-19T23:11:20.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89bd21700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd89407e780 0x7fd894092ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:20.642 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89bd21700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd894072060 con 0x7fd89408f930 2020-03-19T23:11:20.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89bd21700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89407dd50 con 0x7fd89407e780 2020-03-19T23:11:20.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd89407e780 0x7fd894092ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:20.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd89407e780 0x7fd894092ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32816/0 (socket says 172.21.15.38:32816) 2020-03-19T23:11:20.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 -- 172.21.15.38:0/3339789084 learned_addr learned my addr 172.21.15.38:0/3339789084 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:20.643 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd899abd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd89408f930 0x7fd8940708b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44190/0 (socket says 172.21.15.38:44190) 2020-03-19T23:11:20.645 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 -- 172.21.15.38:0/3339789084 >> v1:172.21.15.38:6789/0 conn(0x7fd89408f930 legacy=0x7fd8940708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:20.646 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 -- 172.21.15.38:0/3339789084 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8940939e0 con 0x7fd89407e780 2020-03-19T23:11:20.646 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967167677 0 0) 0x7fd894072060 con 0x7fd89408f930 2020-03-19T23:11:20.646 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> v2:172.21.15.38:3300/0 conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.646 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> v2:172.21.15.38:3300/0 conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=READY pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd890005c10 con 0x7fd89407e780 2020-03-19T23:11:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd890005d70 con 0x7fd89407e780 2020-03-19T23:11:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89407d9c0 con 0x7fd89407e780 2020-03-19T23:11:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd8900061f0 con 0x7fd89407e780 2020-03-19T23:11:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.636+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd894076f20 con 0x7fd89407e780 2020-03-19T23:11:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.637+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.637+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fd890006460 con 0x7fd89407e780 2020-03-19T23:11:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.637+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.637+0000 7fd88affd700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 0x7fd87c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.637+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fd890034160 con 0x7fd89407e780 2020-03-19T23:11:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.637+0000 7fd899abd700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 0x7fd87c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.638+0000 7fd899abd700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 0x7fd87c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd884002ed0 tx=0x7fd884007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.638+0000 7fd899abd700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 0x7fd87c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd884002ed0 tx=0x7fd884007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.638+0000 7fd899abd700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 0x7fd87c02a740 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7fd884002ed0 tx=0x7fd884007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.638+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd894056db0 con 0x7fd89407e780 2020-03-19T23:11:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.641+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:20.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.641+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fd890011610 con 0x7fd89407e780 2020-03-19T23:11:20.794 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:20.791+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_period", "val": "3600"} v 0) v1 -- 0x7fd8940607f0 con 0x7fd89407e780 2020-03-19T23:11:21.379 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.377+0000 7fd89a2be700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=THROTTLE_DONE pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.379 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.377+0000 7fd88affd700 1 -- 172.21.15.38:0/3339789084 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "hit_set_period", "val": "3600"}]=0 set pool 4 hit_set_period to 3600 v23) v1 ==== 155+0+0 (secure 0 0 0) 0x7fd890010fc0 con 0x7fd89407e780 2020-03-19T23:11:21.381 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 msgr2=0x7fd87c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:21.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 0x7fd87c02a740 secure :-1 s=READY pgs=57 cs=0 l=1 rx=0x7fd884002ed0 tx=0x7fd884007a40).stop 2020-03-19T23:11:21.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 msgr2=0x7fd894092ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:21.382 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 secure :-1 s=READY pgs=225 cs=0 l=1 rx=0x7fd890000b80 tx=0x7fd890010990).stop 2020-03-19T23:11:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 shutdown_connections 2020-03-19T23:11:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd89407e780 0x7fd894092ec0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 --2- 172.21.15.38:0/3339789084 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd87c028280 0x7fd87c02a740 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 >> 172.21.15.38:0/3339789084 conn(0x7fd894095460 msgr2=0x7fd8940635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 shutdown_connections 2020-03-19T23:11:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.380+0000 7fd89bd21700 1 -- 172.21.15.38:0/3339789084 wait complete. 2020-03-19T23:11:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 hit_set_period to 3600 2020-03-19T23:11:21.395 INFO:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool set data_cache min_read_recency_for_promote 0' 2020-03-19T23:11:21.518 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.513+0000 7f5ab3168700 1 Processor -- start 2020-03-19T23:11:21.518 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.513+0000 7f5ab3168700 1 -- start start 2020-03-19T23:11:21.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab3168700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5aac08f930 0x7f5aac08fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:21.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab3168700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aac05c700 con 0x7f5aac07e780 2020-03-19T23:11:21.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab3168700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aac090520 con 0x7f5aac08f930 2020-03-19T23:11:21.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab0f04700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5aac08f930 0x7f5aac08fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:21.519 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab1705700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5aac07e780 0x7f5aac08f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44194/0 (socket says 172.21.15.38:44194) 2020-03-19T23:11:21.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab0f04700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5aac08f930 0x7f5aac08fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32822/0 (socket says 172.21.15.38:32822) 2020-03-19T23:11:21.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab1705700 1 -- 172.21.15.38:0/3865099440 learned_addr learned my addr 172.21.15.38:0/3865099440 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:21.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab0f04700 1 -- 172.21.15.38:0/3865099440 >> v1:172.21.15.38:6789/0 conn(0x7f5aac07e780 legacy=0x7f5aac08f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:21.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab0f04700 1 -- 172.21.15.38:0/3865099440 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac07eb80 con 0x7f5aac08f930 2020-03-19T23:11:21.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/3865099440 >> v2:172.21.15.38:3300/0 conn(0x7f5aac08f930 0x7f5aac08fdb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f5a9c002f30 tx=0x7f5a9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/3865099440 >> v2:172.21.15.38:3300/0 conn(0x7f5aac08f930 0x7f5aac08fdb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f5a9c002f30 tx=0x7f5a9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.514+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/3865099440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac08f930 0x7f5aac08fdb0 secure :-1 s=READY pgs=227 cs=0 l=1 rx=0x7f5a9c002f30 tx=0x7f5a9c007a40).ready entity=mon.0 client_cookie=a3c8560a5bf2b214 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:21.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/3865099440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac08f930 0x7f5aac08fdb0 secure :-1 s=THROTTLE_DONE pgs=227 cs=0 l=1 rx=0x7f5a9c002f30 tx=0x7f5a9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5aa3fff700 1 -- 172.21.15.38:0/3865099440 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5a9c0113f0 con 0x7f5aac08f930 2020-03-19T23:11:21.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/3865099440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac08f930 0x7f5aac08fdb0 secure :-1 s=THROTTLE_DONE pgs=227 cs=0 l=1 rx=0x7f5a9c002f30 tx=0x7f5a9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5aa3fff700 1 -- 172.21.15.38:0/3865099440 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c011550 con 0x7f5aac08f930 2020-03-19T23:11:21.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/3865099440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac08f930 0x7f5aac08fdb0 secure :-1 s=THROTTLE_DONE pgs=227 cs=0 l=1 rx=0x7f5a9c002f30 tx=0x7f5a9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5aa3fff700 1 -- 172.21.15.38:0/3865099440 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5a9c011870 con 0x7f5aac08f930 2020-03-19T23:11:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab3168700 1 -- 172.21.15.38:0/3865099440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac08f930 msgr2=0x7f5aac08fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab3168700 1 --2- 172.21.15.38:0/3865099440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac08f930 0x7f5aac08fdb0 secure :-1 s=READY pgs=227 cs=0 l=1 rx=0x7f5a9c002f30 tx=0x7f5a9c007a40).stop 2020-03-19T23:11:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab3168700 1 -- 172.21.15.38:0/3865099440 shutdown_connections 2020-03-19T23:11:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab3168700 1 --2- 172.21.15.38:0/3865099440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac08f930 0x7f5aac08fdb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab3168700 1 -- 172.21.15.38:0/3865099440 >> 172.21.15.38:0/3865099440 conn(0x7f5aac095460 msgr2=0x7f5aac091320 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:21.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab3168700 1 -- 172.21.15.38:0/3865099440 shutdown_connections 2020-03-19T23:11:21.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.515+0000 7f5ab3168700 1 -- 172.21.15.38:0/3865099440 wait complete. 2020-03-19T23:11:21.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.516+0000 7f5ab3168700 1 Processor -- start 2020-03-19T23:11:21.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.516+0000 7f5ab3168700 1 -- start start 2020-03-19T23:11:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab3168700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5aac07e780 0x7f5aac113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab3168700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aac06e1d0 con 0x7f5aac08f930 2020-03-19T23:11:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab3168700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aac062250 con 0x7f5aac07e780 2020-03-19T23:11:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5aac07e780 0x7f5aac113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab0f04700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5aac08f930 0x7f5aac113f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44200/0 (socket says 172.21.15.38:44200) 2020-03-19T23:11:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab0f04700 1 -- 172.21.15.38:0/2407306149 learned_addr learned my addr 172.21.15.38:0/2407306149 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> v2:172.21.15.38:3300/0 conn(0x7f5aac07e780 0x7f5aac113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32826/0 (socket says 172.21.15.38:32826) 2020-03-19T23:11:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 -- 172.21.15.38:0/2407306149 >> v1:172.21.15.38:6789/0 conn(0x7f5aac08f930 legacy=0x7f5aac113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:21.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459562243 0 0) 0x7f5aac06e1d0 con 0x7f5aac08f930 2020-03-19T23:11:21.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 -- 172.21.15.38:0/2407306149 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac1137f0 con 0x7f5aac07e780 2020-03-19T23:11:21.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> v2:172.21.15.38:3300/0 conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> v2:172.21.15.38:3300/0 conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.528 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=READY pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5aa8005bc0 con 0x7f5aac07e780 2020-03-19T23:11:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa8005d20 con 0x7f5aac07e780 2020-03-19T23:11:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac113240 con 0x7f5aac07e780 2020-03-19T23:11:21.530 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f5aa8005e90 con 0x7f5aac07e780 2020-03-19T23:11:21.530 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.517+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac062a30 con 0x7f5aac07e780 2020-03-19T23:11:21.530 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.518+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.530 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.518+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f5aa800a320 con 0x7f5aac07e780 2020-03-19T23:11:21.530 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.518+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.518+0000 7f5aa1ffb700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 0x7f5a8c02a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.518+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f5aa80341d0 con 0x7f5aac07e780 2020-03-19T23:11:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.518+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 0x7f5a8c02a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.518+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 0x7f5a8c02a6a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f5a9c0101c0 tx=0x7f5a9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.519+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac056db0 con 0x7f5aac07e780 2020-03-19T23:11:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.519+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 0x7f5a8c02a6a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f5a9c0101c0 tx=0x7f5a9c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.519+0000 7f5ab0f04700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 0x7f5a8c02a6a0 secure :-1 s=READY pgs=58 cs=0 l=1 rx=0x7f5a9c0101c0 tx=0x7f5a9c007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.522+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.522+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f5aa8007520 con 0x7f5aac07e780 2020-03-19T23:11:21.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:21.671+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "data_cache", "var": "min_read_recency_for_promote", "val": "0"} v 0) v1 -- 0x7f5aac091f00 con 0x7f5aac07e780 2020-03-19T23:11:22.392 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.390+0000 7f5ab1705700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=THROTTLE_DONE pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.393 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.390+0000 7f5aa1ffb700 1 -- 172.21.15.38:0/2407306149 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "data_cache", "var": "min_read_recency_for_promote", "val": "0"}]=0 set pool 4 min_read_recency_for_promote to 0 v24) v1 ==== 177+0+0 (secure 0 0 0) 0x7f5aa8011d40 con 0x7f5aac07e780 2020-03-19T23:11:22.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 msgr2=0x7f5a8c02a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 0x7f5a8c02a6a0 secure :-1 s=READY pgs=58 cs=0 l=1 rx=0x7f5a9c0101c0 tx=0x7f5a9c007a40).stop 2020-03-19T23:11:22.395 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 msgr2=0x7f5aac113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 secure :-1 s=READY pgs=229 cs=0 l=1 rx=0x7f5aac094710 tx=0x7f5aa80108a0).stop 2020-03-19T23:11:22.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 shutdown_connections 2020-03-19T23:11:22.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5aac07e780 0x7f5aac113b20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:22.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 --2- 172.21.15.38:0/2407306149 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f5a8c0281e0 0x7f5a8c02a6a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:22.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 >> 172.21.15.38:0/2407306149 conn(0x7f5aac095460 msgr2=0x7f5aac092e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:22.396 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 shutdown_connections 2020-03-19T23:11:22.397 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.393+0000 7f5ab3168700 1 -- 172.21.15.38:0/2407306149 wait complete. 2020-03-19T23:11:22.397 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 min_read_recency_for_promote to 0 2020-03-19T23:11:22.409 INFO:teuthology.run_tasks:Running task ceph-fuse... 2020-03-19T23:11:22.432 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2020-03-19T23:11:22.432 INFO:tasks.ceph_fuse:config is None 2020-03-19T23:11:22.432 INFO:tasks.ceph_fuse:new config is {'client.1': None, 'client.0': None} 2020-03-19T23:11:22.432 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2020-03-19T23:11:22.433 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:22.555 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.553+0000 7f66aeb7b700 1 Processor -- start 2020-03-19T23:11:22.556 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.553+0000 7f66aeb7b700 1 -- start start 2020-03-19T23:11:22.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66aeb7b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66a8125ba0 0x7f66a8127f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:22.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66aeb7b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a8090af0 con 0x7f66a8065d30 2020-03-19T23:11:22.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66aeb7b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a8066820 con 0x7f66a8125ba0 2020-03-19T23:11:22.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66a7fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66a8125ba0 0x7f66a8127f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:22.558 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66ac917700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f66a8065d30 0x7f66a8066110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44204/0 (socket says 172.21.15.38:44204) 2020-03-19T23:11:22.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66ac917700 1 -- 172.21.15.38:0/470786230 learned_addr learned my addr 172.21.15.38:0/470786230 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:22.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66a7fff700 1 -- 172.21.15.38:0/470786230 >> v1:172.21.15.38:6789/0 conn(0x7f66a8065d30 legacy=0x7f66a8066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66a7fff700 1 -- 172.21.15.38:0/470786230 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a808eec0 con 0x7f66a8125ba0 2020-03-19T23:11:22.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66a77fe700 1 -- 172.21.15.38:0/470786230 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002251162 0 0) 0x7f66a8090af0 con 0x7f66a8065d30 2020-03-19T23:11:22.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66a7fff700 1 --2- 172.21.15.38:0/470786230 >> v2:172.21.15.38:3300/0 conn(0x7f66a8125ba0 0x7f66a8127f90 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6694002f30 tx=0x7f6694007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66a7fff700 1 --2- 172.21.15.38:0/470786230 >> v2:172.21.15.38:3300/0 conn(0x7f66a8125ba0 0x7f66a8127f90 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6694002f30 tx=0x7f6694007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.559 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.554+0000 7f66a7fff700 1 --2- 172.21.15.38:0/470786230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8125ba0 0x7f66a8127f90 secure :-1 s=READY pgs=231 cs=0 l=1 rx=0x7f6694002f30 tx=0x7f6694007a40).ready entity=mon.0 client_cookie=781458cb9f5390bd server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:22.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66a7fff700 1 --2- 172.21.15.38:0/470786230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8125ba0 0x7f66a8127f90 secure :-1 s=THROTTLE_DONE pgs=231 cs=0 l=1 rx=0x7f6694002f30 tx=0x7f6694007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66a77fe700 1 -- 172.21.15.38:0/470786230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6694008e80 con 0x7f66a8125ba0 2020-03-19T23:11:22.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66a7fff700 1 --2- 172.21.15.38:0/470786230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8125ba0 0x7f66a8127f90 secure :-1 s=THROTTLE_DONE pgs=231 cs=0 l=1 rx=0x7f6694002f30 tx=0x7f6694007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66a77fe700 1 -- 172.21.15.38:0/470786230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66940113f0 con 0x7f66a8125ba0 2020-03-19T23:11:22.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66a7fff700 1 --2- 172.21.15.38:0/470786230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8125ba0 0x7f66a8127f90 secure :-1 s=THROTTLE_DONE pgs=231 cs=0 l=1 rx=0x7f6694002f30 tx=0x7f6694007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66a77fe700 1 -- 172.21.15.38:0/470786230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f66940116d0 con 0x7f66a8125ba0 2020-03-19T23:11:22.560 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66aeb7b700 1 -- 172.21.15.38:0/470786230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8125ba0 msgr2=0x7f66a8127f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66aeb7b700 1 --2- 172.21.15.38:0/470786230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8125ba0 0x7f66a8127f90 secure :-1 s=READY pgs=231 cs=0 l=1 rx=0x7f6694002f30 tx=0x7f6694007a40).stop 2020-03-19T23:11:22.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66aeb7b700 1 -- 172.21.15.38:0/470786230 shutdown_connections 2020-03-19T23:11:22.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66aeb7b700 1 --2- 172.21.15.38:0/470786230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8125ba0 0x7f66a8127f90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:22.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66aeb7b700 1 -- 172.21.15.38:0/470786230 >> 172.21.15.38:0/470786230 conn(0x7f66a8091540 msgr2=0x7f66a8091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:22.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66aeb7b700 1 -- 172.21.15.38:0/470786230 shutdown_connections 2020-03-19T23:11:22.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.555+0000 7f66aeb7b700 1 -- 172.21.15.38:0/470786230 wait complete. 2020-03-19T23:11:22.561 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.556+0000 7f66aeb7b700 1 Processor -- start 2020-03-19T23:11:22.562 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.556+0000 7f66aeb7b700 1 -- start start 2020-03-19T23:11:22.564 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.556+0000 7f66aeb7b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66a8065d30 0x7f66a808bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:22.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.556+0000 7f66aeb7b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a808cab0 con 0x7f66a8125ba0 2020-03-19T23:11:22.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.556+0000 7f66aeb7b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a81209d0 con 0x7f66a8065d30 2020-03-19T23:11:22.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66a8065d30 0x7f66a808bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:22.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66a7fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f66a8125ba0 0x7f66a811d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44210/0 (socket says 172.21.15.38:44210) 2020-03-19T23:11:22.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66a8065d30 0x7f66a808bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32836/0 (socket says 172.21.15.38:32836) 2020-03-19T23:11:22.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66a7fff700 1 -- 172.21.15.38:0/485564116 learned_addr learned my addr 172.21.15.38:0/485564116 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:22.565 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 -- 172.21.15.38:0/485564116 >> v1:172.21.15.38:6789/0 conn(0x7f66a8125ba0 legacy=0x7f66a811d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701176072 0 0) 0x7f66a808cab0 con 0x7f66a8125ba0 2020-03-19T23:11:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 -- 172.21.15.38:0/485564116 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8125700 con 0x7f66a8065d30 2020-03-19T23:11:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> v2:172.21.15.38:3300/0 conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> v2:172.21.15.38:3300/0 conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=READY pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f669c005bc0 con 0x7f66a8065d30 2020-03-19T23:11:22.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c005d20 con 0x7f66a8065d30 2020-03-19T23:11:22.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a811eec0 con 0x7f66a8065d30 2020-03-19T23:11:22.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f669c0061f0 con 0x7f66a8065d30 2020-03-19T23:11:22.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.557+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a811fa60 con 0x7f66a8065d30 2020-03-19T23:11:22.567 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f669c006350 con 0x7f66a8065d30 2020-03-19T23:11:22.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66a5ffb700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 0x7f669002a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:22.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f669c0340a0 con 0x7f66a8065d30 2020-03-19T23:11:22.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66a7fff700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 0x7f669002a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:22.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a8056db0 con 0x7f66a8065d30 2020-03-19T23:11:22.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66a7fff700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 0x7f669002a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f66940101c0 tx=0x7f6694007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66a7fff700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 0x7f669002a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f66940101c0 tx=0x7f6694007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.558+0000 7f66a7fff700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 0x7f669002a710 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f66940101c0 tx=0x7f6694007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:22.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.562+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.562+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f669c007ba0 con 0x7f66a8065d30 2020-03-19T23:11:22.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.710+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66a811f050 con 0x7f66a8065d30 2020-03-19T23:11:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.711+0000 7f66ac917700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=THROTTLE_DONE pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.711+0000 7f66a5ffb700 1 -- 172.21.15.38:0/485564116 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10340 (secure 0 0 0) 0x7f669c007160 con 0x7f66a8065d30 2020-03-19T23:11:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:22.714 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:22.382484+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[4],"tier_of":-1,"read_tier":4,"write_tier":4,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"data_cache","create_time":"2020-03-19T23:11:14.491957+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":3,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":3600,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 msgr2=0x7f669002a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 0x7f669002a710 secure :-1 s=READY pgs=59 cs=0 l=1 rx=0x7f66940101c0 tx=0x7f6694007a40).stop 2020-03-19T23:11:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 msgr2=0x7f66a808bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 secure :-1 s=READY pgs=233 cs=0 l=1 rx=0x7f669c000b80 tx=0x7f669c010950).stop 2020-03-19T23:11:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 shutdown_connections 2020-03-19T23:11:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a8065d30 0x7f66a808bef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 --2- 172.21.15.38:0/485564116 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6690028250 0x7f669002a710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 >> 172.21.15.38:0/485564116 conn(0x7f66a8091540 msgr2=0x7f66a8093500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 shutdown_connections 2020-03-19T23:11:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.712+0000 7f66aeb7b700 1 -- 172.21.15.38:0/485564116 wait complete. 2020-03-19T23:11:22.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:10:59.540840+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'12', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:11:02.421156+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_metadata', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 2, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'cephfs': {u'metadata': u'cephfs'}}, u'options': {u'pg_autoscale_bias': 4, u'recovery_priority': 5, u'pg_num_min': 16}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'15', u'min_write_recency_for_promote': 0, u'read_tier': -1}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:11:03.595719+0000', u'snap_epoch': 18, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'cephfs_data', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': 4, u'last_force_op_resend_preluminous': u'20', u'pool': 3, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [4], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'20', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'20', u'application_metadata': {u'cephfs': {u'data': u'cephfs'}}, u'options': {}, u'hit_set_count': 0, u'flags': 1, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'20', u'min_write_recency_for_promote': 0, u'read_tier': 4}, {u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,incomplete_clones', u'tier_of': 3, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 4, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2020-03-19T23:11:14.491957+0000', u'snap_epoch': 18, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 4, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 3600, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 4, u'pg_num': 4, u'type': 1, u'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], u'pool_name': u'data_cache', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 4, u'cache_mode': u'writeback', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'20', u'pool': 4, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'target_size': 0, u'seed': 0, u'type': u'bloom', u'false_positive_probability': 0.05}, u'last_force_op_resend': u'20', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'20', u'application_metadata': {}, u'options': {}, u'hit_set_count': 8, u'flags': 9, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'24', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2020-03-19T23:11:22.726 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-03-19T23:11:22.841 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.839+0000 7f657b1f0700 1 Processor -- start 2020-03-19T23:11:22.842 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.839+0000 7f657b1f0700 1 -- start start 2020-03-19T23:11:22.846 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f657b1f0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6574125060 0x7f6574127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:22.847 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f657b1f0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6574060f10 con 0x7f6574065970 2020-03-19T23:11:22.847 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f657b1f0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6574061490 con 0x7f6574125060 2020-03-19T23:11:22.847 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6574125060 0x7f6574127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:22.847 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6578f8c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6574065970 0x7f6574065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44214/0 (socket says 172.21.15.38:44214) 2020-03-19T23:11:22.847 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6578f8c700 1 -- 172.21.15.38:0/2578223648 learned_addr learned my addr 172.21.15.38:0/2578223648 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:22.847 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 --2- 172.21.15.38:0/2578223648 >> v2:172.21.15.38:3300/0 conn(0x7f6574125060 0x7f6574127450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32842/0 (socket says 172.21.15.38:32842) 2020-03-19T23:11:22.847 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 -- 172.21.15.38:0/2578223648 >> v1:172.21.15.38:6789/0 conn(0x7f6574065970 legacy=0x7f6574065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.848 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 -- 172.21.15.38:0/2578223648 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6574061730 con 0x7f6574125060 2020-03-19T23:11:22.848 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f65737fe700 1 -- 172.21.15.38:0/2578223648 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263205524 0 0) 0x7f65640010d0 con 0x7f6574065970 2020-03-19T23:11:22.848 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 --2- 172.21.15.38:0/2578223648 >> v2:172.21.15.38:3300/0 conn(0x7f6574125060 0x7f6574127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6568002f30 tx=0x7f6568007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.848 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 --2- 172.21.15.38:0/2578223648 >> v2:172.21.15.38:3300/0 conn(0x7f6574125060 0x7f6574127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6568002f30 tx=0x7f6568007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.848 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 --2- 172.21.15.38:0/2578223648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574125060 0x7f6574127450 secure :-1 s=READY pgs=235 cs=0 l=1 rx=0x7f6568002f30 tx=0x7f6568007a40).ready entity=mon.0 client_cookie=d67e7ef2bb8a77c0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:22.848 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.840+0000 7f6573fff700 1 --2- 172.21.15.38:0/2578223648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574125060 0x7f6574127450 secure :-1 s=THROTTLE_DONE pgs=235 cs=0 l=1 rx=0x7f6568002f30 tx=0x7f6568007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.848 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f6573fff700 1 --2- 172.21.15.38:0/2578223648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574125060 0x7f6574127450 secure :-1 s=THROTTLE_DONE pgs=235 cs=0 l=1 rx=0x7f6568002f30 tx=0x7f6568007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.849 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f65737fe700 1 -- 172.21.15.38:0/2578223648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f65680113f0 con 0x7f6574125060 2020-03-19T23:11:22.849 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f6573fff700 1 --2- 172.21.15.38:0/2578223648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574125060 0x7f6574127450 secure :-1 s=THROTTLE_DONE pgs=235 cs=0 l=1 rx=0x7f6568002f30 tx=0x7f6568007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f65737fe700 1 -- 172.21.15.38:0/2578223648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568011550 con 0x7f6574125060 2020-03-19T23:11:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f65737fe700 1 -- 172.21.15.38:0/2578223648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6568011830 con 0x7f6574125060 2020-03-19T23:11:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f657b1f0700 1 -- 172.21.15.38:0/2578223648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574125060 msgr2=0x7f6574127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f657b1f0700 1 --2- 172.21.15.38:0/2578223648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574125060 0x7f6574127450 secure :-1 s=READY pgs=235 cs=0 l=1 rx=0x7f6568002f30 tx=0x7f6568007a40).stop 2020-03-19T23:11:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f657b1f0700 1 -- 172.21.15.38:0/2578223648 shutdown_connections 2020-03-19T23:11:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f657b1f0700 1 --2- 172.21.15.38:0/2578223648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574125060 0x7f6574127450 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f657b1f0700 1 -- 172.21.15.38:0/2578223648 >> 172.21.15.38:0/2578223648 conn(0x7f6574091390 msgr2=0x7f6574091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f657b1f0700 1 -- 172.21.15.38:0/2578223648 shutdown_connections 2020-03-19T23:11:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.841+0000 7f657b1f0700 1 -- 172.21.15.38:0/2578223648 wait complete. 2020-03-19T23:11:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.842+0000 7f657b1f0700 1 Processor -- start 2020-03-19T23:11:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.842+0000 7f657b1f0700 1 -- start start 2020-03-19T23:11:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.842+0000 7f657b1f0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6574065970 0x7f6574124a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.842+0000 7f657b1f0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6574133950 con 0x7f6574125060 2020-03-19T23:11:22.851 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.842+0000 7f657b1f0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6574110cb0 con 0x7f6574065970 2020-03-19T23:11:22.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.842+0000 7f6578f8c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6574065970 0x7f6574124a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:22.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6573fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6574125060 0x7f6574133240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44218/0 (socket says 172.21.15.38:44218) 2020-03-19T23:11:22.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6574065970 0x7f6574124a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32844/0 (socket says 172.21.15.38:32844) 2020-03-19T23:11:22.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6573fff700 1 -- 172.21.15.38:0/4184708364 learned_addr learned my addr 172.21.15.38:0/4184708364 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:22.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 -- 172.21.15.38:0/4184708364 >> v1:172.21.15.38:6789/0 conn(0x7f6574125060 legacy=0x7f6574133240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:22.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 -- 172.21.15.38:0/4184708364 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6574124eb0 con 0x7f6574065970 2020-03-19T23:11:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973503418 0 0) 0x7f6574133950 con 0x7f6574125060 2020-03-19T23:11:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> v2:172.21.15.38:3300/0 conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> v2:172.21.15.38:3300/0 conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=READY pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6564005bc0 con 0x7f6574065970 2020-03-19T23:11:22.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564005d20 con 0x7f6574065970 2020-03-19T23:11:22.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f65640060c0 con 0x7f6574065970 2020-03-19T23:11:22.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65741116c0 con 0x7f6574065970 2020-03-19T23:11:22.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6574129ca0 con 0x7f6574065970 2020-03-19T23:11:22.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.843+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f6564006330 con 0x7f6574065970 2020-03-19T23:11:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6571ffb700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 0x7f655402a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f6564033ec0 con 0x7f6574065970 2020-03-19T23:11:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6574110920 con 0x7f6574065970 2020-03-19T23:11:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6573fff700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 0x7f655402a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6573fff700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 0x7f655402a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f65680101c0 tx=0x7f6568007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6573fff700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 0x7f655402a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f65680101c0 tx=0x7f6568007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.844+0000 7f6573fff700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 0x7f655402a710 secure :-1 s=READY pgs=60 cs=0 l=1 rx=0x7f65680101c0 tx=0x7f6568007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.848+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.848+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f65640090b0 con 0x7f6574065970 2020-03-19T23:11:22.999 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.997+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6574060f10 con 0x7f6574065970 2020-03-19T23:11:22.999 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.997+0000 7f6578f8c700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=THROTTLE_DONE pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:22.999 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.997+0000 7f6571ffb700 1 -- 172.21.15.38:0/4184708364 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6574060f10 con 0x7f6574065970 2020-03-19T23:11:23.000 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:23.001 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 msgr2=0x7f655402a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.001 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 0x7f655402a710 secure :-1 s=READY pgs=60 cs=0 l=1 rx=0x7f65680101c0 tx=0x7f6568007a40).stop 2020-03-19T23:11:23.001 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 msgr2=0x7f6574124a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.002 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 secure :-1 s=READY pgs=237 cs=0 l=1 rx=0x7f6564000b80 tx=0x7f6564010910).stop 2020-03-19T23:11:23.002 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 shutdown_connections 2020-03-19T23:11:23.002 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6574065970 0x7f6574124a50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.002 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 --2- 172.21.15.38:0/4184708364 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f6554028250 0x7f655402a710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.002 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 >> 172.21.15.38:0/4184708364 conn(0x7f6574091390 msgr2=0x7f6574092a50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.002 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 shutdown_connections 2020-03-19T23:11:23.003 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:22.999+0000 7f657b1f0700 1 -- 172.21.15.38:0/4184708364 wait complete. 2020-03-19T23:11:23.011 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_metadata pg_num 2020-03-19T23:11:23.127 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.125+0000 7f648151b700 1 Processor -- start 2020-03-19T23:11:23.128 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.125+0000 7f648151b700 1 -- start start 2020-03-19T23:11:23.132 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f648151b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f647c1224c0 0x7f647c122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f648151b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c1228a0 con 0x7f647c122f40 2020-03-19T23:11:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f648151b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c092200 con 0x7f647c1224c0 2020-03-19T23:11:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f647affd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f647c1224c0 0x7f647c122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f647a7fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f647c122f40 0x7f647c060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44224/0 (socket says 172.21.15.38:44224) 2020-03-19T23:11:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f647a7fc700 1 -- 172.21.15.38:0/18501112 learned_addr learned my addr 172.21.15.38:0/18501112 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f647affd700 1 -- 172.21.15.38:0/18501112 >> v1:172.21.15.38:6789/0 conn(0x7f647c122f40 legacy=0x7f647c060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f647affd700 1 -- 172.21.15.38:0/18501112 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c090b60 con 0x7f647c1224c0 2020-03-19T23:11:23.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.126+0000 7f647affd700 1 --2- 172.21.15.38:0/18501112 >> v2:172.21.15.38:3300/0 conn(0x7f647c1224c0 0x7f647c122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f6464002f30 tx=0x7f6464007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f647affd700 1 --2- 172.21.15.38:0/18501112 >> v2:172.21.15.38:3300/0 conn(0x7f647c1224c0 0x7f647c122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f6464002f30 tx=0x7f6464007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f647affd700 1 --2- 172.21.15.38:0/18501112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1224c0 0x7f647c122ae0 secure :-1 s=READY pgs=239 cs=0 l=1 rx=0x7f6464002f30 tx=0x7f6464007a40).ready entity=mon.0 client_cookie=c638968bc16842a3 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f647affd700 1 --2- 172.21.15.38:0/18501112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1224c0 0x7f647c122ae0 secure :-1 s=THROTTLE_DONE pgs=239 cs=0 l=1 rx=0x7f6464002f30 tx=0x7f6464007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.134 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f647affd700 1 --2- 172.21.15.38:0/18501112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1224c0 0x7f647c122ae0 secure :-1 s=THROTTLE_DONE pgs=239 cs=0 l=1 rx=0x7f6464002f30 tx=0x7f6464007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f6479ffb700 1 -- 172.21.15.38:0/18501112 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6464008e80 con 0x7f647c1224c0 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f647affd700 1 --2- 172.21.15.38:0/18501112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1224c0 0x7f647c122ae0 secure :-1 s=THROTTLE_DONE pgs=239 cs=0 l=1 rx=0x7f6464002f30 tx=0x7f6464007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f6479ffb700 1 -- 172.21.15.38:0/18501112 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64640113f0 con 0x7f647c1224c0 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f6479ffb700 1 -- 172.21.15.38:0/18501112 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f6464011710 con 0x7f647c1224c0 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f648151b700 1 -- 172.21.15.38:0/18501112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1224c0 msgr2=0x7f647c122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f648151b700 1 --2- 172.21.15.38:0/18501112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1224c0 0x7f647c122ae0 secure :-1 s=READY pgs=239 cs=0 l=1 rx=0x7f6464002f30 tx=0x7f6464007a40).stop 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f648151b700 1 -- 172.21.15.38:0/18501112 shutdown_connections 2020-03-19T23:11:23.135 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f648151b700 1 --2- 172.21.15.38:0/18501112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c1224c0 0x7f647c122ae0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f648151b700 1 -- 172.21.15.38:0/18501112 >> 172.21.15.38:0/18501112 conn(0x7f647c091390 msgr2=0x7f647c091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f648151b700 1 -- 172.21.15.38:0/18501112 shutdown_connections 2020-03-19T23:11:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.127+0000 7f648151b700 1 -- 172.21.15.38:0/18501112 wait complete. 2020-03-19T23:11:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.128+0000 7f648151b700 1 Processor -- start 2020-03-19T23:11:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.128+0000 7f648151b700 1 -- start start 2020-03-19T23:11:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.128+0000 7f648151b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f647c122f40 0x7f647c08f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.128+0000 7f648151b700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c10caa0 con 0x7f647c1224c0 2020-03-19T23:11:23.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.128+0000 7f648151b700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c131180 con 0x7f647c122f40 2020-03-19T23:11:23.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647affd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f647c1224c0 0x7f647c133310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44226/0 (socket says 172.21.15.38:44226) 2020-03-19T23:11:23.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647affd700 1 -- 172.21.15.38:0/2539433249 learned_addr learned my addr 172.21.15.38:0/2539433249 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:23.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f647c122f40 0x7f647c08f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 -- 172.21.15.38:0/2539433249 >> v1:172.21.15.38:6789/0 conn(0x7f647c1224c0 legacy=0x7f647c133310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568089285 0 0) 0x7f647c10caa0 con 0x7f647c1224c0 2020-03-19T23:11:23.137 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 -- 172.21.15.38:0/2539433249 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c112650 con 0x7f647c122f40 2020-03-19T23:11:23.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> v2:172.21.15.38:3300/0 conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> v2:172.21.15.38:3300/0 conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=READY pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f646c005bc0 con 0x7f647c122f40 2020-03-19T23:11:23.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.138 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c005d20 con 0x7f647c122f40 2020-03-19T23:11:23.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c130b10 con 0x7f647c122f40 2020-03-19T23:11:23.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f646c005e90 con 0x7f647c122f40 2020-03-19T23:11:23.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.129+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c08fdb0 con 0x7f647c122f40 2020-03-19T23:11:23.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.139 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f646c00a320 con 0x7f647c122f40 2020-03-19T23:11:23.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f6473fff700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 0x7f646002a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f646c034000 con 0x7f647c122f40 2020-03-19T23:11:23.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f647affd700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 0x7f646002a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c111c30 con 0x7f647c122f40 2020-03-19T23:11:23.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f647affd700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 0x7f646002a6a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f6464010180 tx=0x7f6464007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f647affd700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 0x7f646002a6a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f6464010180 tx=0x7f6464007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.140 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.130+0000 7f647affd700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 0x7f646002a6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7f6464010180 tx=0x7f6464007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.141 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.134+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.141 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.134+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f646c0090b0 con 0x7f647c122f40 2020-03-19T23:11:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.282+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f647c10cc30 con 0x7f647c122f40 2020-03-19T23:11:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.282+0000 7f647a7fc700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=THROTTLE_DONE pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.282+0000 7f6473fff700 1 -- 172.21.15.38:0/2539433249 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+10 (secure 0 0 0) 0x7f647c10cc30 con 0x7f647c122f40 2020-03-19T23:11:23.285 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:23.285 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 msgr2=0x7f646002a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.285 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 0x7f646002a6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rx=0x7f6464010180 tx=0x7f6464007a40).stop 2020-03-19T23:11:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 msgr2=0x7f647c08f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 secure :-1 s=READY pgs=241 cs=0 l=1 rx=0x7f647c08e750 tx=0x7f646c0108a0).stop 2020-03-19T23:11:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 shutdown_connections 2020-03-19T23:11:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f64600281e0 0x7f646002a6a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 --2- 172.21.15.38:0/2539433249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f647c122f40 0x7f647c08f290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.286 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 >> 172.21.15.38:0/2539433249 conn(0x7f647c091390 msgr2=0x7f647c093e00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 shutdown_connections 2020-03-19T23:11:23.287 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.283+0000 7f648151b700 1 -- 172.21.15.38:0/2539433249 wait complete. 2020-03-19T23:11:23.296 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get cephfs_data pg_num 2020-03-19T23:11:23.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.409+0000 7fd0bf619700 1 Processor -- start 2020-03-19T23:11:23.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bf619700 1 -- start start 2020-03-19T23:11:23.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bf619700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:23.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bf619700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b8060f10 con 0x7fd0b8065970 2020-03-19T23:11:23.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bf619700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b8061490 con 0x7fd0b8125060 2020-03-19T23:11:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bcbb4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bd3b5700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd0b8065970 0x7fd0b8065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44232/0 (socket says 172.21.15.38:44232) 2020-03-19T23:11:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bcbb4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8127450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32860/0 (socket says 172.21.15.38:32860) 2020-03-19T23:11:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.410+0000 7fd0bd3b5700 1 -- 172.21.15.38:0/1678527136 learned_addr learned my addr 172.21.15.38:0/1678527136 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 -- 172.21.15.38:0/1678527136 >> v1:172.21.15.38:6789/0 conn(0x7fd0b8065970 legacy=0x7fd0b8065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 -- 172.21.15.38:0/1678527136 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8061730 con 0x7fd0b8125060 2020-03-19T23:11:23.415 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0a7fff700 1 -- 172.21.15.38:0/1678527136 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166566880 0 0) 0x7fd0a80010d0 con 0x7fd0b8065970 2020-03-19T23:11:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1678527136 >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1678527136 >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1678527136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8127450 secure :-1 s=READY pgs=243 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).ready entity=mon.0 client_cookie=b09e74ad2385035d server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1678527136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8127450 secure :-1 s=THROTTLE_DONE pgs=243 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0a7fff700 1 -- 172.21.15.38:0/1678527136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd0b4012500 con 0x7fd0b8125060 2020-03-19T23:11:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1678527136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8127450 secure :-1 s=THROTTLE_DONE pgs=243 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.416 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1678527136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8127450 secure :-1 s=THROTTLE_DONE pgs=243 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0a7fff700 1 -- 172.21.15.38:0/1678527136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4012660 con 0x7fd0b8125060 2020-03-19T23:11:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0a7fff700 1 -- 172.21.15.38:0/1678527136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd0b4012980 con 0x7fd0b8125060 2020-03-19T23:11:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bf619700 1 -- 172.21.15.38:0/1678527136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 msgr2=0x7fd0b8127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bf619700 1 --2- 172.21.15.38:0/1678527136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8127450 secure :-1 s=READY pgs=243 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).stop 2020-03-19T23:11:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bf619700 1 -- 172.21.15.38:0/1678527136 shutdown_connections 2020-03-19T23:11:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bf619700 1 --2- 172.21.15.38:0/1678527136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8127450 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.417 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bf619700 1 -- 172.21.15.38:0/1678527136 >> 172.21.15.38:0/1678527136 conn(0x7fd0b8091390 msgr2=0x7fd0b8091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.411+0000 7fd0bf619700 1 -- 172.21.15.38:0/1678527136 shutdown_connections 2020-03-19T23:11:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.412+0000 7fd0bf619700 1 -- 172.21.15.38:0/1678527136 wait complete. 2020-03-19T23:11:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.412+0000 7fd0bf619700 1 Processor -- start 2020-03-19T23:11:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.412+0000 7fd0bf619700 1 -- start start 2020-03-19T23:11:23.420 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bf619700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8131260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bf619700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b81308b0 con 0x7fd0b8065970 2020-03-19T23:11:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bf619700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b8112ca0 con 0x7fd0b8125060 2020-03-19T23:11:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bd3b5700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd0b8065970 0x7fd0b8130b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44236/0 (socket says 172.21.15.38:44236) 2020-03-19T23:11:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bd3b5700 1 -- 172.21.15.38:0/1261683519 learned_addr learned my addr 172.21.15.38:0/1261683519 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8131260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.421 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 -- 172.21.15.38:0/1261683519 >> v1:172.21.15.38:6789/0 conn(0x7fd0b8065970 legacy=0x7fd0b8130b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063681101 0 0) 0x7fd0b81308b0 con 0x7fd0b8065970 2020-03-19T23:11:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 -- 172.21.15.38:0/1261683519 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8111480 con 0x7fd0b8125060 2020-03-19T23:11:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> v2:172.21.15.38:3300/0 conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=READY pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd0b40129e0 con 0x7fd0b8125060 2020-03-19T23:11:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4007150 con 0x7fd0b8125060 2020-03-19T23:11:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b81316c0 con 0x7fd0b8125060 2020-03-19T23:11:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fd0b4007430 con 0x7fd0b8125060 2020-03-19T23:11:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.413+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b810f9d0 con 0x7fd0b8125060 2020-03-19T23:11:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fd0b400f0c0 con 0x7fd0b8125060 2020-03-19T23:11:23.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0a67fc700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 0x7fd09802a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fd0b40348c0 con 0x7fd0b8125060 2020-03-19T23:11:23.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0bd3b5700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 0x7fd09802a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0bd3b5700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 0x7fd09802a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fd0a8005500 tx=0x7fd0a80016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.424 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0bd3b5700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 0x7fd09802a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fd0a8005500 tx=0x7fd0a80016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.414+0000 7fd0bd3b5700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 0x7fd09802a710 secure :-1 s=READY pgs=62 cs=0 l=1 rx=0x7fd0a8005500 tx=0x7fd0a80016e0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.415+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b8056db0 con 0x7fd0b8125060 2020-03-19T23:11:23.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.418+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.425 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.418+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fd0b400fd20 con 0x7fd0b8125060 2020-03-19T23:11:23.569 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.567+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0b8067160 con 0x7fd0b8125060 2020-03-19T23:11:23.569 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.567+0000 7fd0bcbb4700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=THROTTLE_DONE pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.570 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.567+0000 7fd0a67fc700 1 -- 172.21.15.38:0/1261683519 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd0b400fb40 con 0x7fd0b8125060 2020-03-19T23:11:23.570 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 8 2020-03-19T23:11:23.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.568+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 msgr2=0x7fd09802a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.568+0000 7fd0bf619700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 0x7fd09802a710 secure :-1 s=READY pgs=62 cs=0 l=1 rx=0x7fd0a8005500 tx=0x7fd0a80016e0).stop 2020-03-19T23:11:23.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.568+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 msgr2=0x7fd0b8131260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.568+0000 7fd0bf619700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 secure :-1 s=READY pgs=245 cs=0 l=1 rx=0x7fd0b4002f30 tx=0x7fd0b4007a40).stop 2020-03-19T23:11:23.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.569+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 shutdown_connections 2020-03-19T23:11:23.571 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.569+0000 7fd0bf619700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fd098028250 0x7fd09802a710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.569+0000 7fd0bf619700 1 --2- 172.21.15.38:0/1261683519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0b8125060 0x7fd0b8131260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.569+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 >> 172.21.15.38:0/1261683519 conn(0x7fd0b8091390 msgr2=0x7fd0b808dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.569+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 shutdown_connections 2020-03-19T23:11:23.572 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.569+0000 7fd0bf619700 1 -- 172.21.15.38:0/1261683519 wait complete. 2020-03-19T23:11:23.581 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get data_cache pg_num 2020-03-19T23:11:23.697 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.695+0000 7f72df83d700 1 Processor -- start 2020-03-19T23:11:23.697 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.695+0000 7f72df83d700 1 -- start start 2020-03-19T23:11:23.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.695+0000 7f72df83d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:23.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.695+0000 7f72df83d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d81228a0 con 0x7f72d8122f40 2020-03-19T23:11:23.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.695+0000 7f72df83d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d8092200 con 0x7f72d81224c0 2020-03-19T23:11:23.702 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.709 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dcdd8700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f72d8122f40 0x7f72d8060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44244/0 (socket says 172.21.15.38:44244) 2020-03-19T23:11:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8122ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32870/0 (socket says 172.21.15.38:32870) 2020-03-19T23:11:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dcdd8700 1 -- 172.21.15.38:0/4242824784 learned_addr learned my addr 172.21.15.38:0/4242824784 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 -- 172.21.15.38:0/4242824784 >> v1:172.21.15.38:6789/0 conn(0x7f72d8122f40 legacy=0x7f72d8060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 -- 172.21.15.38:0/4242824784 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8090b60 con 0x7f72d81224c0 2020-03-19T23:11:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/4242824784 >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f72d0002f30 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/4242824784 >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f72d0002f30 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/4242824784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8122ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rx=0x7f72d0002f30 tx=0x7f72d0007a40).ready entity=mon.0 client_cookie=c3fa0d3c9b112280 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/4242824784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8122ae0 secure :-1 s=THROTTLE_DONE pgs=247 cs=0 l=1 rx=0x7f72d0002f30 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72d7fff700 1 -- 172.21.15.38:0/4242824784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72d0013730 con 0x7f72d81224c0 2020-03-19T23:11:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/4242824784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8122ae0 secure :-1 s=THROTTLE_DONE pgs=247 cs=0 l=1 rx=0x7f72d0002f30 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/4242824784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8122ae0 secure :-1 s=THROTTLE_DONE pgs=247 cs=0 l=1 rx=0x7f72d0002f30 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72d7fff700 1 -- 172.21.15.38:0/4242824784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0013890 con 0x7f72d81224c0 2020-03-19T23:11:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72d7fff700 1 -- 172.21.15.38:0/4242824784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72d0013bf0 con 0x7f72d81224c0 2020-03-19T23:11:23.711 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72df83d700 1 -- 172.21.15.38:0/4242824784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 msgr2=0x7f72d8122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.696+0000 7f72df83d700 1 --2- 172.21.15.38:0/4242824784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8122ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rx=0x7f72d0002f30 tx=0x7f72d0007a40).stop 2020-03-19T23:11:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.697+0000 7f72df83d700 1 -- 172.21.15.38:0/4242824784 shutdown_connections 2020-03-19T23:11:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.697+0000 7f72df83d700 1 --2- 172.21.15.38:0/4242824784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8122ae0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.697+0000 7f72df83d700 1 -- 172.21.15.38:0/4242824784 >> 172.21.15.38:0/4242824784 conn(0x7f72d8091390 msgr2=0x7f72d8091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.697+0000 7f72df83d700 1 -- 172.21.15.38:0/4242824784 shutdown_connections 2020-03-19T23:11:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.697+0000 7f72df83d700 1 -- 172.21.15.38:0/4242824784 wait complete. 2020-03-19T23:11:23.712 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.697+0000 7f72df83d700 1 Processor -- start 2020-03-19T23:11:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.697+0000 7f72df83d700 1 -- start start 2020-03-19T23:11:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72df83d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8132fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72df83d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d8132cb0 con 0x7f72d8122f40 2020-03-19T23:11:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72df83d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d811ff20 con 0x7f72d81224c0 2020-03-19T23:11:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8132fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8132fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32874/0 (socket says 172.21.15.38:32874) 2020-03-19T23:11:23.713 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dcdd8700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f72d8122f40 0x7f72d8133420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44248/0 (socket says 172.21.15.38:44248) 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 -- 172.21.15.38:0/1427166727 learned_addr learned my addr 172.21.15.38:0/1427166727 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 -- 172.21.15.38:0/1427166727 >> v1:172.21.15.38:6789/0 conn(0x7f72d8122f40 legacy=0x7f72d8133420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110769203 0 0) 0x7f72d8132cb0 con 0x7f72d8122f40 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 -- 172.21.15.38:0/1427166727 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d81126d0 con 0x7f72d81224c0 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> v2:172.21.15.38:3300/0 conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.714 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.698+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72d0011a60 con 0x7f72d81224c0 2020-03-19T23:11:23.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0008970 con 0x7f72d81224c0 2020-03-19T23:11:23.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d81103d0 con 0x7f72d81224c0 2020-03-19T23:11:23.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f72d0008c10 con 0x7f72d81224c0 2020-03-19T23:11:23.715 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d8111040 con 0x7f72d81224c0 2020-03-19T23:11:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f72d0007170 con 0x7f72d81224c0 2020-03-19T23:11:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72d67fc700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 0x7f72b802a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f72d0034240 con 0x7f72d81224c0 2020-03-19T23:11:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.699+0000 7f72dcdd8700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 0x7f72b802a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.700+0000 7f72dcdd8700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 0x7f72b802a6f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f72cc000f00 tx=0x7f72cc001940).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.700+0000 7f72dcdd8700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 0x7f72b802a6f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f72cc000f00 tx=0x7f72cc001940).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.700+0000 7f72dcdd8700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 0x7f72b802a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f72cc000f00 tx=0x7f72cc001940).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.700+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d8090b60 con 0x7f72d81224c0 2020-03-19T23:11:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.703+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.703+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f72d8090b60 con 0x7f72d81224c0 2020-03-19T23:11:23.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.849+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_cache", "var": "pg_num"} v 0) v1 -- 0x7f72d8090b60 con 0x7f72d81224c0 2020-03-19T23:11:23.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.850+0000 7f72dd5d9700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=THROTTLE_DONE pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.852 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.850+0000 7f72d67fc700 1 -- 172.21.15.38:0/1427166727 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_cache", "var": "pg_num"}]=0 v24) v1 ==== 99+0+10 (secure 0 0 0) 0x7f72d8090b60 con 0x7f72d81224c0 2020-03-19T23:11:23.852 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 4 2020-03-19T23:11:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 msgr2=0x7f72b802a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 0x7f72b802a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rx=0x7f72cc000f00 tx=0x7f72cc001940).stop 2020-03-19T23:11:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 msgr2=0x7f72d8132fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 secure :-1 s=READY pgs=249 cs=0 l=1 rx=0x7f72d8132bd0 tx=0x7f72d0007a40).stop 2020-03-19T23:11:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 shutdown_connections 2020-03-19T23:11:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f72d81224c0 0x7f72d8132fc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 --2- 172.21.15.38:0/1427166727 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f72b8028230 0x7f72b802a6f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 >> 172.21.15.38:0/1427166727 conn(0x7f72d8091390 msgr2=0x7f72d8093f50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 shutdown_connections 2020-03-19T23:11:23.855 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.851+0000 7f72df83d700 1 -- 172.21.15.38:0/1427166727 wait complete. 2020-03-19T23:11:23.865 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-03-19T23:11:23.866 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early fs dump --format=json 2020-03-19T23:11:23.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb32e282700 1 Processor -- start 2020-03-19T23:11:23.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb32e282700 1 -- start start 2020-03-19T23:11:23.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb32e282700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb3280667d0 0x7fb328066bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:23.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb32e282700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb328062590 con 0x7fb328067010 2020-03-19T23:11:23.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb32e282700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3280907d0 con 0x7fb3280667d0 2020-03-19T23:11:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb327fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb3280667d0 0x7fb328066bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb3277fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb328067010 0x7fb32808bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44252/0 (socket says 172.21.15.38:44252) 2020-03-19T23:11:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb327fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb3280667d0 0x7fb328066bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32880/0 (socket says 172.21.15.38:32880) 2020-03-19T23:11:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb3277fe700 1 -- 172.21.15.38:0/2178705070 learned_addr learned my addr 172.21.15.38:0/2178705070 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb327fff700 1 -- 172.21.15.38:0/2178705070 >> v1:172.21.15.38:6789/0 conn(0x7fb328067010 legacy=0x7fb32808bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb327fff700 1 -- 172.21.15.38:0/2178705070 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32808d290 con 0x7fb3280667d0 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.977+0000 7fb326ffd700 1 -- 172.21.15.38:0/2178705070 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229527572 0 0) 0x7fb328062590 con 0x7fb328067010 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb327fff700 1 --2- 172.21.15.38:0/2178705070 >> v2:172.21.15.38:3300/0 conn(0x7fb3280667d0 0x7fb328066bb0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fb318002f30 tx=0x7fb318007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb327fff700 1 --2- 172.21.15.38:0/2178705070 >> v2:172.21.15.38:3300/0 conn(0x7fb3280667d0 0x7fb328066bb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fb318002f30 tx=0x7fb318007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb327fff700 1 --2- 172.21.15.38:0/2178705070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3280667d0 0x7fb328066bb0 secure :-1 s=READY pgs=251 cs=0 l=1 rx=0x7fb318002f30 tx=0x7fb318007a40).ready entity=mon.0 client_cookie=e59c3373cd7eb78b server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb327fff700 1 --2- 172.21.15.38:0/2178705070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3280667d0 0x7fb328066bb0 secure :-1 s=THROTTLE_DONE pgs=251 cs=0 l=1 rx=0x7fb318002f30 tx=0x7fb318007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb326ffd700 1 -- 172.21.15.38:0/2178705070 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb318008e80 con 0x7fb3280667d0 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb327fff700 1 --2- 172.21.15.38:0/2178705070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3280667d0 0x7fb328066bb0 secure :-1 s=THROTTLE_DONE pgs=251 cs=0 l=1 rx=0x7fb318002f30 tx=0x7fb318007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.983 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb327fff700 1 --2- 172.21.15.38:0/2178705070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3280667d0 0x7fb328066bb0 secure :-1 s=THROTTLE_DONE pgs=251 cs=0 l=1 rx=0x7fb318002f30 tx=0x7fb318007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb326ffd700 1 -- 172.21.15.38:0/2178705070 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3180113f0 con 0x7fb3280667d0 2020-03-19T23:11:23.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb326ffd700 1 -- 172.21.15.38:0/2178705070 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb3180116d0 con 0x7fb3280667d0 2020-03-19T23:11:23.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb32e282700 1 -- 172.21.15.38:0/2178705070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3280667d0 msgr2=0x7fb328066bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb32e282700 1 --2- 172.21.15.38:0/2178705070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3280667d0 0x7fb328066bb0 secure :-1 s=READY pgs=251 cs=0 l=1 rx=0x7fb318002f30 tx=0x7fb318007a40).stop 2020-03-19T23:11:23.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb32e282700 1 -- 172.21.15.38:0/2178705070 shutdown_connections 2020-03-19T23:11:23.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb32e282700 1 --2- 172.21.15.38:0/2178705070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb3280667d0 0x7fb328066bb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:23.984 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb32e282700 1 -- 172.21.15.38:0/2178705070 >> 172.21.15.38:0/2178705070 conn(0x7fb328091540 msgr2=0x7fb328091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:23.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb32e282700 1 -- 172.21.15.38:0/2178705070 shutdown_connections 2020-03-19T23:11:23.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.978+0000 7fb32e282700 1 -- 172.21.15.38:0/2178705070 wait complete. 2020-03-19T23:11:23.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.979+0000 7fb32e282700 1 Processor -- start 2020-03-19T23:11:23.985 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.979+0000 7fb32e282700 1 -- start start 2020-03-19T23:11:23.987 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb32e282700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb328067010 0x7fb328122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb32e282700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3280607c0 con 0x7fb3280667d0 2020-03-19T23:11:23.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb32e282700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3280620b0 con 0x7fb328067010 2020-03-19T23:11:23.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb327fff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb3280667d0 0x7fb328060e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44256/0 (socket says 172.21.15.38:44256) 2020-03-19T23:11:23.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb327fff700 1 -- 172.21.15.38:0/1732649745 learned_addr learned my addr 172.21.15.38:0/1732649745 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:23.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> v2:172.21.15.38:3300/0 conn(0x7fb328067010 0x7fb328122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.988 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 -- 172.21.15.38:0/1732649745 >> v1:172.21.15.38:6789/0 conn(0x7fb3280667d0 legacy=0x7fb328060e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:23.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078773749 0 0) 0x7fb3280607c0 con 0x7fb3280667d0 2020-03-19T23:11:23.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 -- 172.21.15.38:0/1732649745 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328123100 con 0x7fb328067010 2020-03-19T23:11:23.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> v2:172.21.15.38:3300/0 conn(0x7fb328067010 0x7fb328122770 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> v2:172.21.15.38:3300/0 conn(0x7fb328067010 0x7fb328122770 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=READY pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.989 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb31c005b20 con 0x7fb328067010 2020-03-19T23:11:23.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb328124420 con 0x7fb328067010 2020-03-19T23:11:23.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c005c80 con 0x7fb328067010 2020-03-19T23:11:23.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb31c0060c0 con 0x7fb328067010 2020-03-19T23:11:23.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.980+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3280617e0 con 0x7fb328067010 2020-03-19T23:11:23.990 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.991 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fb31c006330 con 0x7fb328067010 2020-03-19T23:11:23.991 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.991 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb3257fa700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 0x7fb30802a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:23.991 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fb31c034040 con 0x7fb328067010 2020-03-19T23:11:23.991 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb327fff700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 0x7fb30802a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:23.991 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb328056db0 con 0x7fb328067010 2020-03-19T23:11:23.994 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb327fff700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 0x7fb30802a6a0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb318000b80 tx=0x7fb318007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.994 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb327fff700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 0x7fb30802a6a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb318000b80 tx=0x7fb318007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.994 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.981+0000 7fb327fff700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 0x7fb30802a6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rx=0x7fb318000b80 tx=0x7fb318007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:23.995 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.985+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:23.995 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:23.985+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fb31c006e60 con 0x7fb328067010 2020-03-19T23:11:24.183 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.181+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3280678b0 con 0x7fb328067010 2020-03-19T23:11:24.184 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.182+0000 7fb3277fe700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=THROTTLE_DONE pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:24.184 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.182+0000 7fb3257fa700 1 -- 172.21.15.38:0/1732649745 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1783 (secure 0 0 0) 0x7fb31c011df0 con 0x7fb328067010 2020-03-19T23:11:24.184 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:24.185 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-03-19T23:11:04.634508+0000","modified":"2020-03-19T23:11:08.305742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4202},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4202":{"gid":4202,"name":"a","rank":0,"incarnation":5,"state":"up:active","state_seq":2,"addr":"172.21.15.38:6827/3900686918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3900686918},{"type":"v1","addr":"172.21.15.38:6827","nonce":3900686918}]},"join_fscid":-1,"export_targets":[],"features":4540138292836696063,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-03-19T23:11:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 msgr2=0x7fb30802a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 0x7fb30802a6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rx=0x7fb318000b80 tx=0x7fb318007a40).stop 2020-03-19T23:11:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 msgr2=0x7fb328122770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 secure :-1 s=READY pgs=253 cs=0 l=1 rx=0x7fb31c000b80 tx=0x7fb31c010950).stop 2020-03-19T23:11:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 shutdown_connections 2020-03-19T23:11:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb3080281e0 0x7fb30802a6a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 --2- 172.21.15.38:0/1732649745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb328067010 0x7fb328122770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 >> 172.21.15.38:0/1732649745 conn(0x7fb328091540 msgr2=0x7fb328093680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 shutdown_connections 2020-03-19T23:11:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:24.183+0000 7fb32e282700 1 -- 172.21.15.38:0/1732649745 wait complete. 2020-03-19T23:11:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 6 2020-03-19T23:11:24.296 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {u'session_autoclose': 300, u'balancer': u'', u'up': {u'mds_0': 4202}, u'last_failure_osd_epoch': 0, u'in': [0], u'last_failure': 0, u'max_file_size': 1099511627776, u'explicitly_allowed_features': 0, u'damaged': [], u'tableserver': 0, u'metadata_pool': 2, u'failed': [], u'epoch': 6, u'flags': 18, u'max_mds': 1, u'compat': {u'compat': {}, u'ro_compat': {}, u'incompat': {u'feature_10': u'snaprealm v2', u'feature_8': u'no anchor table', u'feature_9': u'file layout v2', u'feature_2': u'client writeable ranges', u'feature_3': u'default file layouts on dirs', u'feature_1': u'base v0.20', u'feature_6': u'dirfrag is stored in omap', u'feature_4': u'dir inode in separate object', u'feature_5': u'mds uses versioned encoding'}}, u'min_compat_client': u'0 (unknown)', u'data_pools': [3], u'info': {u'gid_4202': {u'export_targets': [], u'name': u'a', u'incarnation': 5, u'state_seq': 2, u'state': u'up:active', u'gid': 4202, u'features': 4540138292836696063, u'rank': 0, u'flags': 0, u'join_fscid': -1, u'addrs': {u'addrvec': [{u'nonce': 3900686918, u'type': u'v2', u'addr': u'172.21.15.38:6826'}, {u'nonce': 3900686918, u'type': u'v1', u'addr': u'172.21.15.38:6827'}]}, u'addr': u'172.21.15.38:6827/3900686918'}}, u'fs_name': u'cephfs', u'created': u'2020-03-19T23:11:04.634508+0000', u'standby_count_wanted': 0, u'enabled': True, u'modified': u'2020-03-19T23:11:08.305742+0000', u'session_timeout': 60, u'stopped': [], u'ever_allowed_features': 0, u'root': 0} 2020-03-19T23:11:24.297 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-03-19T23:11:24.297 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok status 2020-03-19T23:11:24.425 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2020-03-19T23:11:24.426 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3", 2020-03-19T23:11:24.426 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2020-03-19T23:11:24.426 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4202, 2020-03-19T23:11:24.426 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2020-03-19T23:11:24.426 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2020-03-19T23:11:24.426 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 17.119484731, 2020-03-19T23:11:24.427 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 6, 2020-03-19T23:11:24.427 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 24, 2020-03-19T23:11:24.427 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 16, 2020-03-19T23:11:24.427 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 17.408254683999999 2020-03-19T23:11:24.427 INFO:teuthology.orchestra.run.smithi038.stdout:} 2020-03-19T23:11:24.434 INFO:tasks.cephfs.filesystem:_json_asok output: { "cluster_fsid": "a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3", "whoami": 0, "id": 4202, "want_state": "up:active", "state": "up:active", "rank_uptime": 17.119484731, "mdsmap_epoch": 6, "osdmap_epoch": 24, "osdmap_epoch_barrier": 16, "uptime": 17.408254683999999 } 2020-03-19T23:11:24.434 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2020-03-19T23:11:24.434 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2020-03-19T23:11:24.435 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi038.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2020-03-19T23:11:24.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:24.435 INFO:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec mkdir -p /home/ubuntu/cephtest/mnt.1) 2020-03-19T23:11:24.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:24.455 INFO:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-03-19T23:11:24.522 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-03-19T23:11:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: mount point does not exist. 2020-03-19T23:11:24.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:24.526 INFO:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2020-03-19T23:11:24.576 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-03-19T23:11:24.581 INFO:teuthology.orchestra.run.smithi038.stderr:ls: cannot access '/sys/fs/fuse/connections': No such file or directory 2020-03-19T23:11:24.581 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2020-03-19T23:11:24.581 INFO:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2020-03-19T23:11:24.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:24.619 INFO:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-03-19T23:11:24.653 DEBUG:teuthology.orchestra.run:got remote process result: 32 2020-03-19T23:11:24.655 INFO:teuthology.orchestra.run.smithi038.stderr:mount: /sys/fs/fuse/connections: mount point does not exist. 2020-03-19T23:11:24.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:24.655 INFO:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2020-03-19T23:11:24.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi038.stderr:2020-03-19T23:11:24.700+0000 7ff8a7b211c0 -1 init, newargv = 0x55bc16a8eee0 newargc=7 2020-03-19T23:11:24.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi038.stderr:ceph-fuse[23556]: starting ceph client 2020-03-19T23:11:24.710 DEBUG:teuthology.orchestra.run:got remote process result: 2 2020-03-19T23:11:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:ls: cannot access '/sys/fs/fuse/connections': No such file or directory 2020-03-19T23:11:24.735 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi038.stderr:ceph-fuse[23556]: starting fuse 2020-03-19T23:11:25.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:25.814 INFO:teuthology.orchestra.run.smithi038:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2020-03-19T23:11:25.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:25.882 INFO:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2020-03-19T23:11:25.911 INFO:teuthology.orchestra.run.smithi038.stdout:47 2020-03-19T23:11:25.911 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2020-03-19T23:11:25.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-03-19T23:11:25.912 INFO:teuthology.orchestra.run.smithi038:> sudo python3 -c ''"'"' 2020-03-19T23:11:25.912 INFO:teuthology.orchestra.run.smithi038:> import glob 2020-03-19T23:11:25.912 INFO:teuthology.orchestra.run.smithi038:> import re 2020-03-19T23:11:25.912 INFO:teuthology.orchestra.run.smithi038:> import os 2020-03-19T23:11:25.912 INFO:teuthology.orchestra.run.smithi038:> import subprocess 2020-03-19T23:11:25.913 INFO:teuthology.orchestra.run.smithi038:> 2020-03-19T23:11:25.913 INFO:teuthology.orchestra.run.smithi038:> def find_socket(client_name): 2020-03-19T23:11:25.913 INFO:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2020-03-19T23:11:25.913 INFO:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2020-03-19T23:11:25.914 INFO:teuthology.orchestra.run.smithi038:> 2020-03-19T23:11:25.914 INFO:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2020-03-19T23:11:25.914 INFO:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2020-03-19T23:11:25.914 INFO:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2020-03-19T23:11:25.914 INFO:teuthology.orchestra.run.smithi038:> return files[0] 2020-03-19T23:11:25.915 INFO:teuthology.orchestra.run.smithi038:> 2020-03-19T23:11:25.915 INFO:teuthology.orchestra.run.smithi038:> for f in files: 2020-03-19T23:11:25.915 INFO:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2020-03-19T23:11:25.915 INFO:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2020-03-19T23:11:25.915 INFO:teuthology.orchestra.run.smithi038:> return f 2020-03-19T23:11:25.916 INFO:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2020-03-19T23:11:25.916 INFO:teuthology.orchestra.run.smithi038:> 2020-03-19T23:11:25.916 INFO:teuthology.orchestra.run.smithi038:> print(find_socket("client.1")) 2020-03-19T23:11:25.916 INFO:teuthology.orchestra.run.smithi038:> '"'"'' 2020-03-19T23:11:26.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T23:11:26.022 INFO:teuthology.orchestra.run.smithi038.stderr: File "", line 1 2020-03-19T23:11:26.022 INFO:teuthology.orchestra.run.smithi038.stderr: ' 2020-03-19T23:11:26.022 INFO:teuthology.orchestra.run.smithi038.stderr: ^ 2020-03-19T23:11:26.022 INFO:teuthology.orchestra.run.smithi038.stderr:SyntaxError: EOL while scanning string literal 2020-03-19T23:11:26.023 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph_fuse.py", line 140, in task info["mount"].mount(mountpoint=mountpoint, mount_path=mount_path) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 33, in mount return self._mount(mount_path, mount_fs_name, mount_options) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 163, in _mount self.gather_mount_info() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 166, in gather_mount_info status = self.admin_socket(['status']) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 449, in admin_socket ], stdout=StringIO(), timeout=(15*60)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi038 with status 1: 'sudo python3 -c \'\'"\'"\'\nimport glob\nimport re\nimport os\nimport subprocess\n\ndef find_socket(client_name):\n asok_path = "/var/run/ceph/ceph-client.1.*.asok"\n files = glob.glob(asok_path)\n\n # Given a non-glob path, it better be there\n if "*" not in asok_path:\n assert(len(files) == 1)\n return files[0]\n\n for f in files:\n pid = re.match(".*\\.(\\d+)\\.asok$", f).group(1)\n if os.path.exists("/proc/{0}".format(pid)):\n return f\n raise RuntimeError("Client socket {0} not found".format(client_name))\n\nprint(find_socket("client.1"))\n\'"\'"\'\'' 2020-03-19T23:11:26.024 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2020-03-19T23:11:26.041 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2020-03-19T23:11:26.062 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2020-03-19T23:11:26.063 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:26.182 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.179+0000 7f805bfff700 1 Processor -- start 2020-03-19T23:11:26.182 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.179+0000 7f805bfff700 1 -- start start 2020-03-19T23:11:26.189 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:26.189 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805bfff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805c060a40 con 0x7f805c0903e0 2020-03-19T23:11:26.189 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805bfff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c060500 con 0x7f805c090ed0 2020-03-19T23:11:26.190 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805a7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.190 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805affd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f805c0903e0 0x7f805c0907c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44274/0 (socket says 172.21.15.38:44274) 2020-03-19T23:11:26.190 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805affd700 1 -- 172.21.15.38:0/4054915427 learned_addr learned my addr 172.21.15.38:0/4054915427 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:26.190 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805a7fc700 1 -- 172.21.15.38:0/4054915427 >> v1:172.21.15.38:6789/0 conn(0x7f805c0903e0 legacy=0x7f805c0907c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.191 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f805a7fc700 1 -- 172.21.15.38:0/4054915427 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c060730 con 0x7f805c090ed0 2020-03-19T23:11:26.191 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.180+0000 7f8059ffb700 1 -- 172.21.15.38:0/4054915427 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586568341 0 0) 0x7f805c060a40 con 0x7f805c0903e0 2020-03-19T23:11:26.191 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805a7fc700 1 --2- 172.21.15.38:0/4054915427 >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08e720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.191 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805a7fc700 1 --2- 172.21.15.38:0/4054915427 >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08e720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805a7fc700 1 --2- 172.21.15.38:0/4054915427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08e720 secure :-1 s=READY pgs=259 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).ready entity=mon.0 client_cookie=8e8677bb0225aacb server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805a7fc700 1 --2- 172.21.15.38:0/4054915427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08e720 secure :-1 s=THROTTLE_DONE pgs=259 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f8059ffb700 1 -- 172.21.15.38:0/4054915427 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f804c011cd0 con 0x7f805c090ed0 2020-03-19T23:11:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805a7fc700 1 --2- 172.21.15.38:0/4054915427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08e720 secure :-1 s=THROTTLE_DONE pgs=259 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f8059ffb700 1 -- 172.21.15.38:0/4054915427 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c011e30 con 0x7f805c090ed0 2020-03-19T23:11:26.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805a7fc700 1 --2- 172.21.15.38:0/4054915427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08e720 secure :-1 s=THROTTLE_DONE pgs=259 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805bfff700 1 -- 172.21.15.38:0/4054915427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 msgr2=0x7f805c08e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f8059ffb700 1 -- 172.21.15.38:0/4054915427 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f804c00a290 con 0x7f805c090ed0 2020-03-19T23:11:26.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805bfff700 1 --2- 172.21.15.38:0/4054915427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08e720 secure :-1 s=READY pgs=259 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).stop 2020-03-19T23:11:26.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805bfff700 1 -- 172.21.15.38:0/4054915427 shutdown_connections 2020-03-19T23:11:26.193 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805bfff700 1 --2- 172.21.15.38:0/4054915427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08e720 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805bfff700 1 -- 172.21.15.38:0/4054915427 >> 172.21.15.38:0/4054915427 conn(0x7f805c0953a0 msgr2=0x7f805c0914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805bfff700 1 -- 172.21.15.38:0/4054915427 shutdown_connections 2020-03-19T23:11:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.181+0000 7f805bfff700 1 -- 172.21.15.38:0/4054915427 wait complete. 2020-03-19T23:11:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.182+0000 7f805bfff700 1 Processor -- start 2020-03-19T23:11:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.182+0000 7f805bfff700 1 -- start start 2020-03-19T23:11:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.182+0000 7f805bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.182+0000 7f805bfff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805c120410 con 0x7f805c0903e0 2020-03-19T23:11:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.182+0000 7f805bfff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c120640 con 0x7f805c090ed0 2020-03-19T23:11:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805affd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f805c0903e0 0x7f805c11d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44276/0 (socket says 172.21.15.38:44276) 2020-03-19T23:11:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805affd700 1 -- 172.21.15.38:0/912959233 learned_addr learned my addr 172.21.15.38:0/912959233 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 -- 172.21.15.38:0/912959233 >> v1:172.21.15.38:6789/0 conn(0x7f805c0903e0 legacy=0x7f805c11d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809002797 0 0) 0x7f805c120410 con 0x7f805c0903e0 2020-03-19T23:11:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 -- 172.21.15.38:0/912959233 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c08ca60 con 0x7f805c090ed0 2020-03-19T23:11:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> v2:172.21.15.38:3300/0 conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=READY pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f804c005a40 con 0x7f805c090ed0 2020-03-19T23:11:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c005ba0 con 0x7f805c090ed0 2020-03-19T23:11:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c11f050 con 0x7f805c090ed0 2020-03-19T23:11:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f804c005e40 con 0x7f805c090ed0 2020-03-19T23:11:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.183+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c11fc60 con 0x7f805c090ed0 2020-03-19T23:11:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f804c00a6e0 con 0x7f805c090ed0 2020-03-19T23:11:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f8053fff700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 0x7f804002a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f804c034f90 con 0x7f805c090ed0 2020-03-19T23:11:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f805affd700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 0x7f804002a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c056db0 con 0x7f805c090ed0 2020-03-19T23:11:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f805affd700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 0x7f804002a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f8044001260 tx=0x7f8044001720).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.199 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f805affd700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 0x7f804002a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f8044001260 tx=0x7f8044001720).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.199 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.184+0000 7f805affd700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 0x7f804002a6c0 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f8044001260 tx=0x7f8044001720).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.199 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.188+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.200 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.188+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f804c0066c0 con 0x7f805c090ed0 2020-03-19T23:11:26.340 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.338+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f805c06aaf0 con 0x7f805c090ed0 2020-03-19T23:11:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.338+0000 7f805a7fc700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=THROTTLE_DONE pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.338+0000 7f8053fff700 1 -- 172.21.15.38:0/912959233 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10340 (secure 0 0 0) 0x7f804c006070 con 0x7f805c090ed0 2020-03-19T23:11:26.341 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:26.342 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:22.382484+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[4],"tier_of":-1,"read_tier":4,"write_tier":4,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"data_cache","create_time":"2020-03-19T23:11:14.491957+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":3,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":3600,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:26.343 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 msgr2=0x7f804002a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.343 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 0x7f804002a6c0 secure :-1 s=READY pgs=65 cs=0 l=1 rx=0x7f8044001260 tx=0x7f8044001720).stop 2020-03-19T23:11:26.343 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 msgr2=0x7f805c08bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 secure :-1 s=READY pgs=261 cs=0 l=1 rx=0x7f804c0032b0 tx=0x7f804c007dc0).stop 2020-03-19T23:11:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 shutdown_connections 2020-03-19T23:11:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f8040028200 0x7f804002a6c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 --2- 172.21.15.38:0/912959233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f805c090ed0 0x7f805c08bef0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 >> 172.21.15.38:0/912959233 conn(0x7f805c0953a0 msgr2=0x7f805c062bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 shutdown_connections 2020-03-19T23:11:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.340+0000 7f805bfff700 1 -- 172.21.15.38:0/912959233 wait complete. 2020-03-19T23:11:26.353 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T23:11:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.465+0000 7fe545775700 1 Processor -- start 2020-03-19T23:11:26.467 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.465+0000 7fe545775700 1 -- start start 2020-03-19T23:11:26.472 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.465+0000 7fe545775700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe540126dc0 0x7fe5401291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:26.472 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.465+0000 7fe545775700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe540062590 con 0x7fe5400667d0 2020-03-19T23:11:26.472 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.465+0000 7fe545775700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5400907d0 con 0x7fe540126dc0 2020-03-19T23:11:26.472 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.465+0000 7fe53e7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe540126dc0 0x7fe5401291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.473 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.465+0000 7fe53effd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe5400667d0 0x7fe540066bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44282/0 (socket says 172.21.15.38:44282) 2020-03-19T23:11:26.473 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe540126dc0 0x7fe5401291b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32912/0 (socket says 172.21.15.38:32912) 2020-03-19T23:11:26.473 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53effd700 1 -- 172.21.15.38:0/3581432123 learned_addr learned my addr 172.21.15.38:0/3581432123 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:26.473 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 -- 172.21.15.38:0/3581432123 >> v1:172.21.15.38:6789/0 conn(0x7fe5400667d0 legacy=0x7fe540066bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.473 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 -- 172.21.15.38:0/3581432123 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54008d290 con 0x7fe540126dc0 2020-03-19T23:11:26.473 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/3581432123 >> v2:172.21.15.38:3300/0 conn(0x7fe540126dc0 0x7fe5401291b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe5280032b0 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.474 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/3581432123 >> v2:172.21.15.38:3300/0 conn(0x7fe540126dc0 0x7fe5401291b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe5280032b0 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.474 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/3581432123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe540126dc0 0x7fe5401291b0 secure :-1 s=READY pgs=263 cs=0 l=1 rx=0x7fe5280032b0 tx=0x7fe528007dc0).ready entity=mon.0 client_cookie=1e2e51f26d55b8c server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.474 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/3581432123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe540126dc0 0x7fe5401291b0 secure :-1 s=THROTTLE_DONE pgs=263 cs=0 l=1 rx=0x7fe5280032b0 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.474 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53dffb700 1 -- 172.21.15.38:0/3581432123 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe528011cd0 con 0x7fe540126dc0 2020-03-19T23:11:26.474 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/3581432123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe540126dc0 0x7fe5401291b0 secure :-1 s=THROTTLE_DONE pgs=263 cs=0 l=1 rx=0x7fe5280032b0 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.474 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/3581432123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe540126dc0 0x7fe5401291b0 secure :-1 s=THROTTLE_DONE pgs=263 cs=0 l=1 rx=0x7fe5280032b0 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.474 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53dffb700 1 -- 172.21.15.38:0/3581432123 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe528011e30 con 0x7fe540126dc0 2020-03-19T23:11:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe53dffb700 1 -- 172.21.15.38:0/3581432123 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe52800a290 con 0x7fe540126dc0 2020-03-19T23:11:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe545775700 1 -- 172.21.15.38:0/3581432123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe540126dc0 msgr2=0x7fe5401291b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.466+0000 7fe545775700 1 --2- 172.21.15.38:0/3581432123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe540126dc0 0x7fe5401291b0 secure :-1 s=READY pgs=263 cs=0 l=1 rx=0x7fe5280032b0 tx=0x7fe528007dc0).stop 2020-03-19T23:11:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.467+0000 7fe545775700 1 -- 172.21.15.38:0/3581432123 shutdown_connections 2020-03-19T23:11:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.467+0000 7fe545775700 1 --2- 172.21.15.38:0/3581432123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe540126dc0 0x7fe5401291b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.467+0000 7fe545775700 1 -- 172.21.15.38:0/3581432123 >> 172.21.15.38:0/3581432123 conn(0x7fe540091540 msgr2=0x7fe540091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:26.475 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.467+0000 7fe545775700 1 -- 172.21.15.38:0/3581432123 shutdown_connections 2020-03-19T23:11:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.467+0000 7fe545775700 1 -- 172.21.15.38:0/3581432123 wait complete. 2020-03-19T23:11:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.467+0000 7fe545775700 1 Processor -- start 2020-03-19T23:11:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.467+0000 7fe545775700 1 -- start start 2020-03-19T23:11:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe545775700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe5400667d0 0x7fe54011f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe545775700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe540120610 con 0x7fe540126dc0 2020-03-19T23:11:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe545775700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54011efe0 con 0x7fe5400667d0 2020-03-19T23:11:26.476 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe5400667d0 0x7fe54011f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53e7fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe540126dc0 0x7fe540060380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44288/0 (socket says 172.21.15.38:44288) 2020-03-19T23:11:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53e7fc700 1 -- 172.21.15.38:0/903101877 learned_addr learned my addr 172.21.15.38:0/903101877 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499447319 0 0) 0x7fe540120610 con 0x7fe540126dc0 2020-03-19T23:11:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 -- 172.21.15.38:0/903101877 >> v1:172.21.15.38:6789/0 conn(0x7fe540126dc0 legacy=0x7fe540060380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 -- 172.21.15.38:0/903101877 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540061a80 con 0x7fe5400667d0 2020-03-19T23:11:26.477 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> v2:172.21.15.38:3300/0 conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.478 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> v2:172.21.15.38:3300/0 conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.478 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=READY pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.478 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.478 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe538005b20 con 0x7fe5400667d0 2020-03-19T23:11:26.478 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.478 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538005c80 con 0x7fe5400667d0 2020-03-19T23:11:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.468+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe540124560 con 0x7fe5400667d0 2020-03-19T23:11:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe5380060c0 con 0x7fe5400667d0 2020-03-19T23:11:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe540060a90 con 0x7fe5400667d0 2020-03-19T23:11:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fe53800a320 con 0x7fe5400667d0 2020-03-19T23:11:26.479 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe537fff700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fe538034060 con 0x7fe5400667d0 2020-03-19T23:11:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.469+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.470+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54011f150 con 0x7fe5400667d0 2020-03-19T23:11:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.470+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe528003250 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.470+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe528003250 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.480 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.470+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rx=0x7fe528003250 tx=0x7fe528007dc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.473+0000 7fe53effd700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=THROTTLE_DONE pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.482 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.473+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fe54011f150 con 0x7fe5400667d0 2020-03-19T23:11:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.616+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe54011f150 con 0x7fe524028200 2020-03-19T23:11:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.618+0000 7fe53e7fc700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 secure :-1 s=THROTTLE_DONE pgs=66 cs=0 l=1 rx=0x7fe528003250 tx=0x7fe528007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.620 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.618+0000 7fe537fff700 1 -- 172.21.15.38:0/903101877 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+74020 (secure 0 0 0) 0x7fe54011f150 con 0x7fe524028200 2020-03-19T23:11:26.624 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:26.624 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 msgr2=0x7fe52402a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rx=0x7fe528003250 tx=0x7fe528007dc0).stop 2020-03-19T23:11:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 msgr2=0x7fe54011f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 secure :-1 s=READY pgs=265 cs=0 l=1 rx=0x7fe540091300 tx=0x7fe5380108a0).stop 2020-03-19T23:11:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 shutdown_connections 2020-03-19T23:11:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5400667d0 0x7fe54011f950 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 --2- 172.21.15.38:0/903101877 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe524028200 0x7fe52402a6c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 >> 172.21.15.38:0/903101877 conn(0x7fe540091540 msgr2=0x7fe54008be20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 shutdown_connections 2020-03-19T23:11:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.620+0000 7fe545775700 1 -- 172.21.15.38:0/903101877 wait complete. 2020-03-19T23:11:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2020-03-19T23:11:26.632 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2020-03-19T23:11:25.618636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146376,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426424,"statfs":{"total":322122547200,"available":318900658176,"internally_reserved":3221225472,"allocated":638976,"data_stored":216170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000904"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"11","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738799+0000","last_change":"2020-03-19T23:11:16.354394+0000","last_active":"2020-03-19T23:11:19.738799+0000","last_peered":"2020-03-19T23:11:19.738799+0000","last_clean":"2020-03-19T23:11:19.738799+0000","last_became_active":"2020-03-19T23:11:16.353972+0000","last_became_peered":"2020-03-19T23:11:16.353972+0000","last_unstale":"2020-03-19T23:11:19.738799+0000","last_undegraded":"2020-03-19T23:11:19.738799+0000","last_fullsized":"2020-03-19T23:11:19.738799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"14","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.739435+0000","last_change":"2020-03-19T23:11:04.653879+0000","last_active":"2020-03-19T23:11:19.739435+0000","last_peered":"2020-03-19T23:11:19.739435+0000","last_clean":"2020-03-19T23:11:19.739435+0000","last_became_active":"2020-03-19T23:11:04.653521+0000","last_became_peered":"2020-03-19T23:11:04.653521+0000","last_unstale":"2020-03-19T23:11:19.739435+0000","last_undegraded":"2020-03-19T23:11:19.739435+0000","last_fullsized":"2020-03-19T23:11:19.739435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"18","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738929+0000","last_change":"2020-03-19T23:11:01.250472+0000","last_active":"2020-03-19T23:11:19.738929+0000","last_peered":"2020-03-19T23:11:19.738929+0000","last_clean":"2020-03-19T23:11:19.738929+0000","last_became_active":"2020-03-19T23:11:01.250084+0000","last_became_peered":"2020-03-19T23:11:01.250084+0000","last_unstale":"2020-03-19T23:11:19.738929+0000","last_undegraded":"2020-03-19T23:11:19.738929+0000","last_fullsized":"2020-03-19T23:11:19.738929+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'4","reported_seq":"20","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029801+0000","last_change":"2020-03-19T23:11:04.285615+0000","last_active":"2020-03-19T23:11:21.029801+0000","last_peered":"2020-03-19T23:11:21.029801+0000","last_clean":"2020-03-19T23:11:21.029801+0000","last_became_active":"2020-03-19T23:11:04.284829+0000","last_became_peered":"2020-03-19T23:11:04.284829+0000","last_unstale":"2020-03-19T23:11:21.029801+0000","last_undegraded":"2020-03-19T23:11:21.029801+0000","last_fullsized":"2020-03-19T23:11:21.029801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738785+0000","last_change":"2020-03-19T23:11:16.355229+0000","last_active":"2020-03-19T23:11:19.738785+0000","last_peered":"2020-03-19T23:11:19.738785+0000","last_clean":"2020-03-19T23:11:19.738785+0000","last_became_active":"2020-03-19T23:11:16.354846+0000","last_became_peered":"2020-03-19T23:11:16.354846+0000","last_unstale":"2020-03-19T23:11:19.738785+0000","last_undegraded":"2020-03-19T23:11:19.738785+0000","last_fullsized":"2020-03-19T23:11:19.738785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"14","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738826+0000","last_change":"2020-03-19T23:11:04.654105+0000","last_active":"2020-03-19T23:11:19.738826+0000","last_peered":"2020-03-19T23:11:19.738826+0000","last_clean":"2020-03-19T23:11:19.738826+0000","last_became_active":"2020-03-19T23:11:04.653741+0000","last_became_peered":"2020-03-19T23:11:04.653741+0000","last_unstale":"2020-03-19T23:11:19.738826+0000","last_undegraded":"2020-03-19T23:11:19.738826+0000","last_fullsized":"2020-03-19T23:11:19.738826+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029528+0000","last_change":"2020-03-19T23:11:01.251743+0000","last_active":"2020-03-19T23:11:21.029528+0000","last_peered":"2020-03-19T23:11:21.029528+0000","last_clean":"2020-03-19T23:11:21.029528+0000","last_became_active":"2020-03-19T23:11:01.251378+0000","last_became_peered":"2020-03-19T23:11:01.251378+0000","last_unstale":"2020-03-19T23:11:21.029528+0000","last_undegraded":"2020-03-19T23:11:21.029528+0000","last_fullsized":"2020-03-19T23:11:21.029528+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"23","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.030049+0000","last_change":"2020-03-19T23:11:04.285389+0000","last_active":"2020-03-19T23:11:21.030049+0000","last_peered":"2020-03-19T23:11:21.030049+0000","last_clean":"2020-03-19T23:11:21.030049+0000","last_became_active":"2020-03-19T23:11:04.284979+0000","last_became_peered":"2020-03-19T23:11:04.284979+0000","last_unstale":"2020-03-19T23:11:21.030049+0000","last_undegraded":"2020-03-19T23:11:21.030049+0000","last_fullsized":"2020-03-19T23:11:21.030049+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"14","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738968+0000","last_change":"2020-03-19T23:11:04.650535+0000","last_active":"2020-03-19T23:11:19.738968+0000","last_peered":"2020-03-19T23:11:19.738968+0000","last_clean":"2020-03-19T23:11:19.738968+0000","last_became_active":"2020-03-19T23:11:04.649316+0000","last_became_peered":"2020-03-19T23:11:04.649316+0000","last_unstale":"2020-03-19T23:11:19.738968+0000","last_undegraded":"2020-03-19T23:11:19.738968+0000","last_fullsized":"2020-03-19T23:11:19.738968+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029809+0000","last_change":"2020-03-19T23:11:01.248950+0000","last_active":"2020-03-19T23:11:21.029809+0000","last_peered":"2020-03-19T23:11:21.029809+0000","last_clean":"2020-03-19T23:11:21.029809+0000","last_became_active":"2020-03-19T23:11:01.248513+0000","last_became_peered":"2020-03-19T23:11:01.248513+0000","last_unstale":"2020-03-19T23:11:21.029809+0000","last_undegraded":"2020-03-19T23:11:21.029809+0000","last_fullsized":"2020-03-19T23:11:21.029809+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"23","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738696+0000","last_change":"2020-03-19T23:11:04.288163+0000","last_active":"2020-03-19T23:11:19.738696+0000","last_peered":"2020-03-19T23:11:19.738696+0000","last_clean":"2020-03-19T23:11:19.738696+0000","last_became_active":"2020-03-19T23:11:04.287139+0000","last_became_peered":"2020-03-19T23:11:04.287139+0000","last_unstale":"2020-03-19T23:11:19.738696+0000","last_undegraded":"2020-03-19T23:11:19.738696+0000","last_fullsized":"2020-03-19T23:11:19.738696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029691+0000","last_change":"2020-03-19T23:11:04.654111+0000","last_active":"2020-03-19T23:11:21.029691+0000","last_peered":"2020-03-19T23:11:21.029691+0000","last_clean":"2020-03-19T23:11:21.029691+0000","last_became_active":"2020-03-19T23:11:04.653768+0000","last_became_peered":"2020-03-19T23:11:04.653768+0000","last_unstale":"2020-03-19T23:11:21.029691+0000","last_undegraded":"2020-03-19T23:11:21.029691+0000","last_fullsized":"2020-03-19T23:11:21.029691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"18","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.739126+0000","last_change":"2020-03-19T23:11:01.252367+0000","last_active":"2020-03-19T23:11:19.739126+0000","last_peered":"2020-03-19T23:11:19.739126+0000","last_clean":"2020-03-19T23:11:19.739126+0000","last_became_active":"2020-03-19T23:11:01.251474+0000","last_became_peered":"2020-03-19T23:11:01.251474+0000","last_unstale":"2020-03-19T23:11:19.739126+0000","last_undegraded":"2020-03-19T23:11:19.739126+0000","last_fullsized":"2020-03-19T23:11:19.739126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"15","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738946+0000","last_change":"2020-03-19T23:11:04.287593+0000","last_active":"2020-03-19T23:11:19.738946+0000","last_peered":"2020-03-19T23:11:19.738946+0000","last_clean":"2020-03-19T23:11:19.738946+0000","last_became_active":"2020-03-19T23:11:04.286797+0000","last_became_peered":"2020-03-19T23:11:04.286797+0000","last_unstale":"2020-03-19T23:11:19.738946+0000","last_undegraded":"2020-03-19T23:11:19.738946+0000","last_fullsized":"2020-03-19T23:11:19.738946+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029347+0000","last_change":"2020-03-19T23:11:04.650882+0000","last_active":"2020-03-19T23:11:21.029347+0000","last_peered":"2020-03-19T23:11:21.029347+0000","last_clean":"2020-03-19T23:11:21.029347+0000","last_became_active":"2020-03-19T23:11:04.650546+0000","last_became_peered":"2020-03-19T23:11:04.650546+0000","last_unstale":"2020-03-19T23:11:21.029347+0000","last_undegraded":"2020-03-19T23:11:21.029347+0000","last_fullsized":"2020-03-19T23:11:21.029347+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029628+0000","last_change":"2020-03-19T23:11:01.249741+0000","last_active":"2020-03-19T23:11:21.029628+0000","last_peered":"2020-03-19T23:11:21.029628+0000","last_clean":"2020-03-19T23:11:21.029628+0000","last_became_active":"2020-03-19T23:11:01.249356+0000","last_became_peered":"2020-03-19T23:11:01.249356+0000","last_unstale":"2020-03-19T23:11:21.029628+0000","last_undegraded":"2020-03-19T23:11:21.029628+0000","last_fullsized":"2020-03-19T23:11:21.029628+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"21","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029698+0000","last_change":"2020-03-19T23:11:04.284232+0000","last_active":"2020-03-19T23:11:21.029698+0000","last_peered":"2020-03-19T23:11:21.029698+0000","last_clean":"2020-03-19T23:11:21.029698+0000","last_became_active":"2020-03-19T23:11:04.283884+0000","last_became_peered":"2020-03-19T23:11:04.283884+0000","last_unstale":"2020-03-19T23:11:21.029698+0000","last_undegraded":"2020-03-19T23:11:21.029698+0000","last_fullsized":"2020-03-19T23:11:21.029698+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"14","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.739242+0000","last_change":"2020-03-19T23:11:04.652133+0000","last_active":"2020-03-19T23:11:19.739242+0000","last_peered":"2020-03-19T23:11:19.739242+0000","last_clean":"2020-03-19T23:11:19.739242+0000","last_became_active":"2020-03-19T23:11:04.651761+0000","last_became_peered":"2020-03-19T23:11:04.651761+0000","last_unstale":"2020-03-19T23:11:19.739242+0000","last_undegraded":"2020-03-19T23:11:19.739242+0000","last_fullsized":"2020-03-19T23:11:19.739242+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"18","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738887+0000","last_change":"2020-03-19T23:11:01.250680+0000","last_active":"2020-03-19T23:11:19.738887+0000","last_peered":"2020-03-19T23:11:19.738887+0000","last_clean":"2020-03-19T23:11:19.738887+0000","last_became_active":"2020-03-19T23:11:01.250171+0000","last_became_peered":"2020-03-19T23:11:01.250171+0000","last_unstale":"2020-03-19T23:11:19.738887+0000","last_undegraded":"2020-03-19T23:11:19.738887+0000","last_fullsized":"2020-03-19T23:11:19.738887+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"16","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.739217+0000","last_change":"2020-03-19T23:11:04.287858+0000","last_active":"2020-03-19T23:11:19.739217+0000","last_peered":"2020-03-19T23:11:19.739217+0000","last_clean":"2020-03-19T23:11:19.739217+0000","last_became_active":"2020-03-19T23:11:04.287457+0000","last_became_peered":"2020-03-19T23:11:04.287457+0000","last_unstale":"2020-03-19T23:11:19.739217+0000","last_undegraded":"2020-03-19T23:11:19.739217+0000","last_fullsized":"2020-03-19T23:11:19.739217+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"11","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.739156+0000","last_change":"2020-03-19T23:11:16.353357+0000","last_active":"2020-03-19T23:11:19.739156+0000","last_peered":"2020-03-19T23:11:19.739156+0000","last_clean":"2020-03-19T23:11:19.739156+0000","last_became_active":"2020-03-19T23:11:16.352787+0000","last_became_peered":"2020-03-19T23:11:16.352787+0000","last_unstale":"2020-03-19T23:11:19.739156+0000","last_undegraded":"2020-03-19T23:11:19.739156+0000","last_fullsized":"2020-03-19T23:11:19.739156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029351+0000","last_change":"2020-03-19T23:11:04.654429+0000","last_active":"2020-03-19T23:11:21.029351+0000","last_peered":"2020-03-19T23:11:21.029351+0000","last_clean":"2020-03-19T23:11:21.029351+0000","last_became_active":"2020-03-19T23:11:04.654078+0000","last_became_peered":"2020-03-19T23:11:04.654078+0000","last_unstale":"2020-03-19T23:11:21.029351+0000","last_undegraded":"2020-03-19T23:11:21.029351+0000","last_fullsized":"2020-03-19T23:11:21.029351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029675+0000","last_change":"2020-03-19T23:11:01.252623+0000","last_active":"2020-03-19T23:11:21.029675+0000","last_peered":"2020-03-19T23:11:21.029675+0000","last_clean":"2020-03-19T23:11:21.029675+0000","last_became_active":"2020-03-19T23:11:01.252205+0000","last_became_peered":"2020-03-19T23:11:01.252205+0000","last_unstale":"2020-03-19T23:11:21.029675+0000","last_undegraded":"2020-03-19T23:11:21.029675+0000","last_fullsized":"2020-03-19T23:11:21.029675+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"22","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029629+0000","last_change":"2020-03-19T23:11:04.285188+0000","last_active":"2020-03-19T23:11:21.029629+0000","last_peered":"2020-03-19T23:11:21.029629+0000","last_clean":"2020-03-19T23:11:21.029629+0000","last_became_active":"2020-03-19T23:11:04.284579+0000","last_became_peered":"2020-03-19T23:11:04.284579+0000","last_unstale":"2020-03-19T23:11:21.029629+0000","last_undegraded":"2020-03-19T23:11:21.029629+0000","last_fullsized":"2020-03-19T23:11:21.029629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029548+0000","last_change":"2020-03-19T23:11:16.354554+0000","last_active":"2020-03-19T23:11:21.029548+0000","last_peered":"2020-03-19T23:11:21.029548+0000","last_clean":"2020-03-19T23:11:21.029548+0000","last_became_active":"2020-03-19T23:11:16.354173+0000","last_became_peered":"2020-03-19T23:11:16.354173+0000","last_unstale":"2020-03-19T23:11:21.029548+0000","last_undegraded":"2020-03-19T23:11:21.029548+0000","last_fullsized":"2020-03-19T23:11:21.029548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"14","reported_epoch":"21","state":"active+clean","last_fresh":"2020-03-19T23:11:19.738820+0000","last_change":"2020-03-19T23:11:04.650468+0000","last_active":"2020-03-19T23:11:19.738820+0000","last_peered":"2020-03-19T23:11:19.738820+0000","last_clean":"2020-03-19T23:11:19.738820+0000","last_became_active":"2020-03-19T23:11:04.650124+0000","last_became_peered":"2020-03-19T23:11:04.650124+0000","last_unstale":"2020-03-19T23:11:19.738820+0000","last_undegraded":"2020-03-19T23:11:19.738820+0000","last_fullsized":"2020-03-19T23:11:19.738820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029941+0000","last_change":"2020-03-19T23:11:01.249394+0000","last_active":"2020-03-19T23:11:21.029941+0000","last_peered":"2020-03-19T23:11:21.029941+0000","last_clean":"2020-03-19T23:11:21.029941+0000","last_became_active":"2020-03-19T23:11:01.249024+0000","last_became_peered":"2020-03-19T23:11:01.249024+0000","last_unstale":"2020-03-19T23:11:21.029941+0000","last_undegraded":"2020-03-19T23:11:21.029941+0000","last_fullsized":"2020-03-19T23:11:21.029941+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"16'8","reported_seq":"24","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:21.029594+0000","last_change":"2020-03-19T23:11:04.283517+0000","last_active":"2020-03-19T23:11:21.029594+0000","last_peered":"2020-03-19T23:11:21.029594+0000","last_clean":"2020-03-19T23:11:21.029594+0000","last_became_active":"2020-03-19T23:11:04.283065+0000","last_became_peered":"2020-03-19T23:11:04.283065+0000","last_unstale":"2020-03-19T23:11:21.029594+0000","last_undegraded":"2020-03-19T23:11:21.029594+0000","last_fullsized":"2020-03-19T23:11:21.029594+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1526,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2286,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738376,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048772,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808828,"statfs":{"total":107374182400,"available":106300239872,"internally_reserved":1073741824,"allocated":192512,"data_stored":69123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048796,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808804,"statfs":{"total":107374182400,"available":106300215296,"internally_reserved":1073741824,"allocated":217088,"data_stored":71341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048808,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808792,"statfs":{"total":107374182400,"available":106300203008,"internally_reserved":1073741824,"allocated":229376,"data_stored":75706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]}]}} 2020-03-19T23:11:26.635 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-03-19T23:11:26.747 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.745+0000 7f24843e5700 1 Processor -- start 2020-03-19T23:11:26.748 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.745+0000 7f24843e5700 1 -- start start 2020-03-19T23:11:26.750 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.746+0000 7f24843e5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c066110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:26.750 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.746+0000 7f24843e5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247c090af0 con 0x7f247c066570 2020-03-19T23:11:26.750 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.746+0000 7f24843e5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f247c08eec0 con 0x7f247c065d30 2020-03-19T23:11:26.750 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.746+0000 7f2482181700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c066110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.751 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.746+0000 7f2481980700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f247c066570 0x7f247c08d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44294/0 (socket says 172.21.15.38:44294) 2020-03-19T23:11:26.751 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.746+0000 7f2481980700 1 -- 172.21.15.38:0/666576003 learned_addr learned my addr 172.21.15.38:0/666576003 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:26.751 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.746+0000 7f2482181700 1 -- 172.21.15.38:0/666576003 >> v1:172.21.15.38:6789/0 conn(0x7f247c066570 legacy=0x7f247c08d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.751 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f248117f700 1 -- 172.21.15.38:0/666576003 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459018588 0 0) 0x7f247c090af0 con 0x7f247c066570 2020-03-19T23:11:26.751 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f2482181700 1 -- 172.21.15.38:0/666576003 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c060ae0 con 0x7f247c065d30 2020-03-19T23:11:26.751 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f2482181700 1 --2- 172.21.15.38:0/666576003 >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c066110 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.751 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f2482181700 1 --2- 172.21.15.38:0/666576003 >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c066110 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f2482181700 1 --2- 172.21.15.38:0/666576003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c066110 secure :-1 s=READY pgs=267 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).ready entity=mon.0 client_cookie=ea17251582b10b8 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f2482181700 1 --2- 172.21.15.38:0/666576003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c066110 secure :-1 s=THROTTLE_DONE pgs=267 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f248117f700 1 -- 172.21.15.38:0/666576003 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2470008e80 con 0x7f247c065d30 2020-03-19T23:11:26.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f2482181700 1 --2- 172.21.15.38:0/666576003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c066110 secure :-1 s=THROTTLE_DONE pgs=267 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f248117f700 1 -- 172.21.15.38:0/666576003 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24700113f0 con 0x7f247c065d30 2020-03-19T23:11:26.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f2482181700 1 --2- 172.21.15.38:0/666576003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c066110 secure :-1 s=THROTTLE_DONE pgs=267 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.752 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f24843e5700 1 -- 172.21.15.38:0/666576003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 msgr2=0x7f247c066110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f24843e5700 1 --2- 172.21.15.38:0/666576003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c066110 secure :-1 s=READY pgs=267 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).stop 2020-03-19T23:11:26.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f248117f700 1 -- 172.21.15.38:0/666576003 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f24700116d0 con 0x7f247c065d30 2020-03-19T23:11:26.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f24843e5700 1 -- 172.21.15.38:0/666576003 shutdown_connections 2020-03-19T23:11:26.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f24843e5700 1 --2- 172.21.15.38:0/666576003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c066110 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f24843e5700 1 -- 172.21.15.38:0/666576003 >> 172.21.15.38:0/666576003 conn(0x7f247c091540 msgr2=0x7f247c091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:26.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f24843e5700 1 -- 172.21.15.38:0/666576003 shutdown_connections 2020-03-19T23:11:26.753 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.747+0000 7f24843e5700 1 -- 172.21.15.38:0/666576003 wait complete. 2020-03-19T23:11:26.754 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.748+0000 7f24843e5700 1 Processor -- start 2020-03-19T23:11:26.754 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.748+0000 7f24843e5700 1 -- start start 2020-03-19T23:11:26.757 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.748+0000 7f24843e5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c082b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:26.757 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.748+0000 7f24843e5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247c084ec0 con 0x7f247c066570 2020-03-19T23:11:26.757 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.748+0000 7f24843e5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f247c0849c0 con 0x7f247c065d30 2020-03-19T23:11:26.757 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c082b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.758 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2481980700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f247c066570 0x7f247c082fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44298/0 (socket says 172.21.15.38:44298) 2020-03-19T23:11:26.758 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2481980700 1 -- 172.21.15.38:0/4253436054 learned_addr learned my addr 172.21.15.38:0/4253436054 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:26.758 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c082b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32924/0 (socket says 172.21.15.38:32924) 2020-03-19T23:11:26.758 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 -- 172.21.15.38:0/4253436054 >> v1:172.21.15.38:6789/0 conn(0x7f247c066570 legacy=0x7f247c082fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.758 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 -- 172.21.15.38:0/4253436054 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c085a30 con 0x7f247c065d30 2020-03-19T23:11:26.758 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067420273 0 0) 0x7f247c084ec0 con 0x7f247c066570 2020-03-19T23:11:26.758 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.759 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> v2:172.21.15.38:3300/0 conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.759 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=READY pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.759 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.759 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f2470011dc0 con 0x7f247c065d30 2020-03-19T23:11:26.759 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.759 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.759 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470007150 con 0x7f247c065d30 2020-03-19T23:11:26.760 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247c083f60 con 0x7f247c065d30 2020-03-19T23:11:26.760 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f24700073f0 con 0x7f247c065d30 2020-03-19T23:11:26.760 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.749+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c08c170 con 0x7f247c065d30 2020-03-19T23:11:26.760 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.760 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f24700056c0 con 0x7f247c065d30 2020-03-19T23:11:26.760 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.760 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f246f7fe700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 0x7f245c02a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:26.761 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f2470034680 con 0x7f247c065d30 2020-03-19T23:11:26.761 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f2481980700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 0x7f245c02a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:26.761 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f2481980700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 0x7f245c02a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f24780012b0 tx=0x7f2478000ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.761 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f2481980700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 0x7f245c02a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f24780012b0 tx=0x7f2478000ef0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.761 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f2481980700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 0x7f245c02a710 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f24780012b0 tx=0x7f2478000ef0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:26.761 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.750+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f247c056db0 con 0x7f247c065d30 2020-03-19T23:11:26.762 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.754+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.762 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.754+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f2470009e60 con 0x7f247c065d30 2020-03-19T23:11:26.906 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.904+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f247c085a30 con 0x7f247c065d30 2020-03-19T23:11:26.907 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.905+0000 7f2482181700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=THROTTLE_DONE pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:26.907 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.905+0000 7f246f7fe700 1 -- 172.21.15.38:0/4253436054 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10340 (secure 0 0 0) 0x7f247000a160 con 0x7f247c065d30 2020-03-19T23:11:26.907 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:26.907 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"a8fe07d0-c3d5-441e-9ffa-5d3fa35e6ba3","created":"2020-03-19T23:10:50.191414+0000","modified":"2020-03-19T23:11:22.382484+0000","last_up_change":"2020-03-19T23:10:56.215570+0000","last_in_change":"2020-03-19T23:10:56.215570+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2020-03-19T23:10:59.540840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2020-03-19T23:11:02.421156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2020-03-19T23:11:03.595719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[4],"tier_of":-1,"read_tier":4,"write_tier":4,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":4,"pool_name":"data_cache","create_time":"2020-03-19T23:11:14.491957+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"20","last_force_op_resend_prenautilus":"20","last_force_op_resend_preluminous":"20","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":3,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":3600,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"e499998d-82cd-4828-9cef-fc02ff95bbd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":20729},{"type":"v1","addr":"172.21.15.38:6809","nonce":20729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":20729},{"type":"v1","addr":"172.21.15.38:6811","nonce":20729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":20729},{"type":"v1","addr":"172.21.15.38:6815","nonce":20729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":20729},{"type":"v1","addr":"172.21.15.38:6813","nonce":20729}]},"public_addr":"172.21.15.38:6809/20729","cluster_addr":"172.21.15.38:6811/20729","heartbeat_back_addr":"172.21.15.38:6815/20729","heartbeat_front_addr":"172.21.15.38:6813/20729","state":["exists","up"]},{"osd":1,"uuid":"28063113-c054-454f-8f2a-d4525cfb896a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":20730},{"type":"v1","addr":"172.21.15.38:6801","nonce":20730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":20730},{"type":"v1","addr":"172.21.15.38:6803","nonce":20730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":20730},{"type":"v1","addr":"172.21.15.38:6807","nonce":20730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":20730},{"type":"v1","addr":"172.21.15.38:6805","nonce":20730}]},"public_addr":"172.21.15.38:6801/20730","cluster_addr":"172.21.15.38:6803/20730","heartbeat_back_addr":"172.21.15.38:6807/20730","heartbeat_front_addr":"172.21.15.38:6805/20730","state":["exists","up"]},{"osd":2,"uuid":"f4403c1d-0ebc-4a48-9e62-aa667c0c7f36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":20739},{"type":"v1","addr":"172.21.15.38:6817","nonce":20739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":20739},{"type":"v1","addr":"172.21.15.38:6819","nonce":20739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":20739},{"type":"v1","addr":"172.21.15.38:6823","nonce":20739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":20739},{"type":"v1","addr":"172.21.15.38:6821","nonce":20739}]},"public_addr":"172.21.15.38:6817/20739","cluster_addr":"172.21.15.38:6819/20739","heartbeat_back_addr":"172.21.15.38:6823/20739","heartbeat_front_addr":"172.21.15.38:6821/20739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.372502+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.343653+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2020-03-19T23:10:54.323753+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-03-19T23:11:26.908 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 msgr2=0x7f245c02a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 0x7f245c02a710 secure :-1 s=READY pgs=67 cs=0 l=1 rx=0x7f24780012b0 tx=0x7f2478000ef0).stop 2020-03-19T23:11:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 msgr2=0x7f247c082b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 secure :-1 s=READY pgs=269 cs=0 l=1 rx=0x7f2470002f30 tx=0x7f2470007a40).stop 2020-03-19T23:11:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 shutdown_connections 2020-03-19T23:11:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f247c065d30 0x7f247c082b50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 --2- 172.21.15.38:0/4253436054 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f245c028250 0x7f245c02a710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 >> 172.21.15.38:0/4253436054 conn(0x7f247c091540 msgr2=0x7f247c093690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:26.910 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 shutdown_connections 2020-03-19T23:11:26.910 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:26.906+0000 7f24843e5700 1 -- 172.21.15.38:0/4253436054 wait complete. 2020-03-19T23:11:27.919 INFO:tasks.ceph:Scrubbing osd.0 2020-03-19T23:11:27.920 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 config set osd_debug_deep_scrub_sleep 0 2020-03-19T23:11:28.050 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.048+0000 7fe83f41e700 1 Processor -- start 2020-03-19T23:11:28.051 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83f41e700 1 -- start start 2020-03-19T23:11:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83f41e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe838065d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83f41e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe838060f10 con 0x7fe8380661b0 2020-03-19T23:11:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83f41e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe838061490 con 0x7fe838065970 2020-03-19T23:11:28.053 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83d1ba700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe838065d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83c9b9700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe8380661b0 0x7fe83808f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44302/0 (socket says 172.21.15.38:44302) 2020-03-19T23:11:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83c9b9700 1 -- 172.21.15.38:0/3332282453 learned_addr learned my addr 172.21.15.38:0/3332282453 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe83d1ba700 1 -- 172.21.15.38:0/3332282453 >> v1:172.21.15.38:6789/0 conn(0x7fe8380661b0 legacy=0x7fe83808f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83d1ba700 1 -- 172.21.15.38:0/3332282453 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838061730 con 0x7fe838065970 2020-03-19T23:11:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.049+0000 7fe827fff700 1 -- 172.21.15.38:0/3332282453 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366266787 0 0) 0x7fe8340010d0 con 0x7fe8380661b0 2020-03-19T23:11:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/3332282453 >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe838065d50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/3332282453 >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe838065d50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/3332282453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe838065d50 secure :-1 s=READY pgs=271 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).ready entity=mon.0 client_cookie=d97986de0d2805bc server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/3332282453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe838065d50 secure :-1 s=THROTTLE_DONE pgs=271 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe827fff700 1 -- 172.21.15.38:0/3332282453 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe8280113f0 con 0x7fe838065970 2020-03-19T23:11:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/3332282453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe838065d50 secure :-1 s=THROTTLE_DONE pgs=271 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe827fff700 1 -- 172.21.15.38:0/3332282453 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe828011550 con 0x7fe838065970 2020-03-19T23:11:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/3332282453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe838065d50 secure :-1 s=THROTTLE_DONE pgs=271 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.055 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe827fff700 1 -- 172.21.15.38:0/3332282453 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe828011830 con 0x7fe838065970 2020-03-19T23:11:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83f41e700 1 -- 172.21.15.38:0/3332282453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 msgr2=0x7fe838065d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83f41e700 1 --2- 172.21.15.38:0/3332282453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe838065d50 secure :-1 s=READY pgs=271 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).stop 2020-03-19T23:11:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83f41e700 1 -- 172.21.15.38:0/3332282453 shutdown_connections 2020-03-19T23:11:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83f41e700 1 --2- 172.21.15.38:0/3332282453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe838065d50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83f41e700 1 -- 172.21.15.38:0/3332282453 >> 172.21.15.38:0/3332282453 conn(0x7fe838091390 msgr2=0x7fe838091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83f41e700 1 -- 172.21.15.38:0/3332282453 shutdown_connections 2020-03-19T23:11:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.050+0000 7fe83f41e700 1 -- 172.21.15.38:0/3332282453 wait complete. 2020-03-19T23:11:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.051+0000 7fe83f41e700 1 Processor -- start 2020-03-19T23:11:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.051+0000 7fe83f41e700 1 -- start start 2020-03-19T23:11:28.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83f41e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe83812e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83f41e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83812e460 con 0x7fe8380661b0 2020-03-19T23:11:28.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83f41e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83812bf10 con 0x7fe838065970 2020-03-19T23:11:28.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe83812e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.062 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83c9b9700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe8380661b0 0x7fe83812eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44308/0 (socket says 172.21.15.38:44308) 2020-03-19T23:11:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83c9b9700 1 -- 172.21.15.38:0/2598113409 learned_addr learned my addr 172.21.15.38:0/2598113409 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 -- 172.21.15.38:0/2598113409 >> v1:172.21.15.38:6789/0 conn(0x7fe8380661b0 legacy=0x7fe83812eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716720984 0 0) 0x7fe83812e460 con 0x7fe8380661b0 2020-03-19T23:11:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 -- 172.21.15.38:0/2598113409 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83810f420 con 0x7fe838065970 2020-03-19T23:11:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> v2:172.21.15.38:3300/0 conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.063 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=READY pgs=273 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe82800f7c0 con 0x7fe838065970 2020-03-19T23:11:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe828007150 con 0x7fe838065970 2020-03-19T23:11:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83812f2d0 con 0x7fe838065970 2020-03-19T23:11:28.064 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.052+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fe8280073f0 con 0x7fe838065970 2020-03-19T23:11:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe838110210 con 0x7fe838065970 2020-03-19T23:11:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fe8280056c0 con 0x7fe838065970 2020-03-19T23:11:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe83d1ba700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=THROTTLE_DONE pgs=273 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe8267fc700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 0x7fe81802a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fe8280348c0 con 0x7fe838065970 2020-03-19T23:11:28.065 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe83c9b9700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 0x7fe81802a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe83f41e700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7fe838055b30 0x7fe838055f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.053+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 --> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe838067160 con 0x7fe838055b30 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.054+0000 7fe83c9b9700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 0x7fe81802a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fe834005500 tx=0x7fe8340016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.054+0000 7fe83d9bb700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7fe838055b30 0x7fe838055f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.054+0000 7fe83c9b9700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 0x7fe81802a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fe834005500 tx=0x7fe8340016e0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.054+0000 7fe83c9b9700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 0x7fe81802a710 secure :-1 s=READY pgs=68 cs=0 l=1 rx=0x7fe834005500 tx=0x7fe8340016e0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.054+0000 7fe83d9bb700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7fe838055b30 0x7fe838055f10 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.066 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.055+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== osd.0 v2:172.21.15.38:6808/20729 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7fe838067160 con 0x7fe838055b30 2020-03-19T23:11:28.073 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.071+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 --> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe838056370 con 0x7fe838055b30 2020-03-19T23:11:28.074 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe8267fc700 1 -- 172.21.15.38:0/2598113409 <== osd.0 v2:172.21.15.38:6808/20729 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe838056370 con 0x7fe838055b30 2020-03-19T23:11:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7fe838055b30 msgr2=0x7fe838055f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7fe838055b30 0x7fe838055f10 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 msgr2=0x7fe81802a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 0x7fe81802a710 secure :-1 s=READY pgs=68 cs=0 l=1 rx=0x7fe834005500 tx=0x7fe8340016e0).stop 2020-03-19T23:11:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 msgr2=0x7fe83812e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 secure :-1 s=READY pgs=273 cs=0 l=1 rx=0x7fe828002f30 tx=0x7fe828007a40).stop 2020-03-19T23:11:28.075 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 shutdown_connections 2020-03-19T23:11:28.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6808/20729,v1:172.21.15.38:6809/20729] conn(0x7fe838055b30 0x7fe838055f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe838065970 0x7fe83812e700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 --2- 172.21.15.38:0/2598113409 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fe818028250 0x7fe81802a710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 >> 172.21.15.38:0/2598113409 conn(0x7fe838091390 msgr2=0x7fe838093070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 shutdown_connections 2020-03-19T23:11:28.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.072+0000 7fe83f41e700 1 -- 172.21.15.38:0/2598113409 wait complete. 2020-03-19T23:11:28.077 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2020-03-19T23:11:28.077 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-03-19T23:11:28.077 INFO:teuthology.orchestra.run.smithi038.stdout:} 2020-03-19T23:11:28.084 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 0 2020-03-19T23:11:28.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb6a6aec700 1 Processor -- start 2020-03-19T23:11:28.198 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb6a6aec700 1 -- start start 2020-03-19T23:11:28.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb6a6aec700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a011f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:28.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb6a6aec700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a00911c0 con 0x7fb6a011fab0 2020-03-19T23:11:28.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb6a6aec700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a0067130 con 0x7fb6a0091eb0 2020-03-19T23:11:28.203 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb6a4888700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a011f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb69ffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb6a011fab0 0x7fb6a0060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44314/0 (socket says 172.21.15.38:44314) 2020-03-19T23:11:28.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.196+0000 7fb69ffff700 1 -- 172.21.15.38:0/232493890 learned_addr learned my addr 172.21.15.38:0/232493890 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 -- 172.21.15.38:0/232493890 >> v1:172.21.15.38:6789/0 conn(0x7fb6a011fab0 legacy=0x7fb6a0060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 -- 172.21.15.38:0/232493890 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a011f340 con 0x7fb6a0091eb0 2020-03-19T23:11:28.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 --2- 172.21.15.38:0/232493890 >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a011f650 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 --2- 172.21.15.38:0/232493890 >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a011f650 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.204 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 --2- 172.21.15.38:0/232493890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a011f650 secure :-1 s=READY pgs=275 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).ready entity=mon.0 client_cookie=ef59e1182e2e2096 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 --2- 172.21.15.38:0/232493890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a011f650 secure :-1 s=THROTTLE_DONE pgs=275 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 --2- 172.21.15.38:0/232493890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a011f650 secure :-1 s=THROTTLE_DONE pgs=275 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb69f7fe700 1 -- 172.21.15.38:0/232493890 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb698010c50 con 0x7fb6a0091eb0 2020-03-19T23:11:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a4888700 1 --2- 172.21.15.38:0/232493890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a011f650 secure :-1 s=THROTTLE_DONE pgs=275 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb69f7fe700 1 -- 172.21.15.38:0/232493890 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb698010db0 con 0x7fb6a0091eb0 2020-03-19T23:11:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb69f7fe700 1 -- 172.21.15.38:0/232493890 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb6980111f0 con 0x7fb6a0091eb0 2020-03-19T23:11:28.205 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a6aec700 1 -- 172.21.15.38:0/232493890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 msgr2=0x7fb6a011f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.197+0000 7fb6a6aec700 1 --2- 172.21.15.38:0/232493890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a011f650 secure :-1 s=READY pgs=275 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).stop 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.198+0000 7fb6a6aec700 1 -- 172.21.15.38:0/232493890 shutdown_connections 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.198+0000 7fb6a6aec700 1 --2- 172.21.15.38:0/232493890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a011f650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.198+0000 7fb6a6aec700 1 -- 172.21.15.38:0/232493890 >> 172.21.15.38:0/232493890 conn(0x7fb6a0091390 msgr2=0x7fb6a00917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.198+0000 7fb6a6aec700 1 -- 172.21.15.38:0/232493890 shutdown_connections 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.198+0000 7fb6a6aec700 1 -- 172.21.15.38:0/232493890 wait complete. 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.198+0000 7fb6a6aec700 1 Processor -- start 2020-03-19T23:11:28.206 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.198+0000 7fb6a6aec700 1 -- start start 2020-03-19T23:11:28.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a6aec700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a00642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a6aec700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a0123b50 con 0x7fb6a011fab0 2020-03-19T23:11:28.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a6aec700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a0065f90 con 0x7fb6a0091eb0 2020-03-19T23:11:28.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a4888700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a00642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb69ffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb6a011fab0 0x7fb6a012fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44320/0 (socket says 172.21.15.38:44320) 2020-03-19T23:11:28.207 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb69ffff700 1 -- 172.21.15.38:0/4045429711 learned_addr learned my addr 172.21.15.38:0/4045429711 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.209 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a4888700 1 -- 172.21.15.38:0/4045429711 >> v1:172.21.15.38:6789/0 conn(0x7fb6a011fab0 legacy=0x7fb6a012fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.209 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895620835 0 0) 0x7fb6a0123b50 con 0x7fb6a011fab0 2020-03-19T23:11:28.209 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a4888700 1 -- 172.21.15.38:0/4045429711 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a010b550 con 0x7fb6a0091eb0 2020-03-19T23:11:28.209 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.209 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> v2:172.21.15.38:3300/0 conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.210 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=READY pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.210 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.199+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.210 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb698011e70 con 0x7fb6a0091eb0 2020-03-19T23:11:28.210 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.210 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.210 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb698005a40 con 0x7fb6a0091eb0 2020-03-19T23:11:28.210 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a0123840 con 0x7fb6a0091eb0 2020-03-19T23:11:28.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fb698005d60 con 0x7fb6a0091eb0 2020-03-19T23:11:28.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a0130520 con 0x7fb6a0091eb0 2020-03-19T23:11:28.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fb69800f280 con 0x7fb6a0091eb0 2020-03-19T23:11:28.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb69dffb700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.211 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fb69800f0c0 con 0x7fb6a0091eb0 2020-03-19T23:11:28.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a0066d00 con 0x7fb6a0091eb0 2020-03-19T23:11:28.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.200+0000 7fb69ffff700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.201+0000 7fb69ffff700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fb694000ee0 tx=0x7fb6940017d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.201+0000 7fb69ffff700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fb694000ee0 tx=0x7fb6940017d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.212 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.201+0000 7fb69ffff700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7fb694000ee0 tx=0x7fb6940017d0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.204+0000 7fb6a4888700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=THROTTLE_DONE pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.213 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.204+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fb69800f510 con 0x7fb6a0091eb0 2020-03-19T23:11:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.356+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6a010b6e0 con 0x7fb6800282d0 2020-03-19T23:11:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.356+0000 7fb69ffff700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 secure :-1 s=THROTTLE_DONE pgs=69 cs=0 l=1 rx=0x7fb694000ee0 tx=0x7fb6940017d0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.356+0000 7fb69dffb700 1 -- 172.21.15.38:0/4045429711 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb6940098a0 con 0x7fb6800282d0 2020-03-19T23:11:28.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.357+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 msgr2=0x7fb68002a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.357+0000 7fb6a6aec700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 secure :-1 s=READY pgs=69 cs=0 l=1 rx=0x7fb694000ee0 tx=0x7fb6940017d0).stop 2020-03-19T23:11:28.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.357+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 msgr2=0x7fb6a00642f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.357+0000 7fb6a6aec700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 secure :-1 s=READY pgs=277 cs=0 l=1 rx=0x7fb6940010d0 tx=0x7fb698007dc0).stop 2020-03-19T23:11:28.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.358+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 shutdown_connections 2020-03-19T23:11:28.360 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.358+0000 7fb6a6aec700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6a0091eb0 0x7fb6a00642f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.358+0000 7fb6a6aec700 1 --2- 172.21.15.38:0/4045429711 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fb6800282d0 0x7fb68002a790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.358+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 >> 172.21.15.38:0/4045429711 conn(0x7fb6a0091390 msgr2=0x7fb6a011e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.358+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 shutdown_connections 2020-03-19T23:11:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.358+0000 7fb6a6aec700 1 -- 172.21.15.38:0/4045429711 wait complete. 2020-03-19T23:11:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 0 to deep-scrub 2020-03-19T23:11:28.370 INFO:tasks.ceph:Scrubbing osd.1 2020-03-19T23:11:28.370 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 config set osd_debug_deep_scrub_sleep 0 2020-03-19T23:11:28.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91b2615700 1 Processor -- start 2020-03-19T23:11:28.487 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91b2615700 1 -- start start 2020-03-19T23:11:28.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91b2615700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:28.511 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91b2615700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac1228a0 con 0x7f91ac122f40 2020-03-19T23:11:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91b2615700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ac092200 con 0x7f91ac1224c0 2020-03-19T23:11:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91abfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91abfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac122ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32952/0 (socket says 172.21.15.38:32952) 2020-03-19T23:11:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91ab7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f91ac122f40 0x7f91ac060110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44324/0 (socket says 172.21.15.38:44324) 2020-03-19T23:11:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.485+0000 7f91abfff700 1 -- 172.21.15.38:0/2868161125 learned_addr learned my addr 172.21.15.38:0/2868161125 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 -- 172.21.15.38:0/2868161125 >> v1:172.21.15.38:6789/0 conn(0x7f91ac122f40 legacy=0x7f91ac060110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 -- 172.21.15.38:0/2868161125 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac090b60 con 0x7f91ac1224c0 2020-03-19T23:11:28.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 --2- 172.21.15.38:0/2868161125 >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac122ae0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 --2- 172.21.15.38:0/2868161125 >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac122ae0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 --2- 172.21.15.38:0/2868161125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac122ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).ready entity=mon.0 client_cookie=40ec2c381ee6b961 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 --2- 172.21.15.38:0/2868161125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac122ae0 secure :-1 s=THROTTLE_DONE pgs=279 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91aaffd700 1 -- 172.21.15.38:0/2868161125 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f919c0114b0 con 0x7f91ac1224c0 2020-03-19T23:11:28.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 --2- 172.21.15.38:0/2868161125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac122ae0 secure :-1 s=THROTTLE_DONE pgs=279 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91abfff700 1 --2- 172.21.15.38:0/2868161125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac122ae0 secure :-1 s=THROTTLE_DONE pgs=279 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91aaffd700 1 -- 172.21.15.38:0/2868161125 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c011610 con 0x7f91ac1224c0 2020-03-19T23:11:28.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91aaffd700 1 -- 172.21.15.38:0/2868161125 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f919c011970 con 0x7f91ac1224c0 2020-03-19T23:11:28.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91b2615700 1 -- 172.21.15.38:0/2868161125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 msgr2=0x7f91ac122ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91b2615700 1 --2- 172.21.15.38:0/2868161125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac122ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).stop 2020-03-19T23:11:28.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91b2615700 1 -- 172.21.15.38:0/2868161125 shutdown_connections 2020-03-19T23:11:28.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91b2615700 1 --2- 172.21.15.38:0/2868161125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac122ae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.486+0000 7f91b2615700 1 -- 172.21.15.38:0/2868161125 >> 172.21.15.38:0/2868161125 conn(0x7f91ac091390 msgr2=0x7f91ac091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.487+0000 7f91b2615700 1 -- 172.21.15.38:0/2868161125 shutdown_connections 2020-03-19T23:11:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.487+0000 7f91b2615700 1 -- 172.21.15.38:0/2868161125 wait complete. 2020-03-19T23:11:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.487+0000 7f91b2615700 1 Processor -- start 2020-03-19T23:11:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.487+0000 7f91b2615700 1 -- start start 2020-03-19T23:11:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91b2615700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac1121f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91b2615700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac130fe0 con 0x7f91ac122f40 2020-03-19T23:11:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91b2615700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ac110cb0 con 0x7f91ac1224c0 2020-03-19T23:11:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac1121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91ab7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f91ac122f40 0x7f91ac133200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44328/0 (socket says 172.21.15.38:44328) 2020-03-19T23:11:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91ab7fe700 1 -- 172.21.15.38:0/2207013616 learned_addr learned my addr 172.21.15.38:0/2207013616 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 -- 172.21.15.38:0/2207013616 >> v1:172.21.15.38:6789/0 conn(0x7f91ac122f40 legacy=0x7f91ac133200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697419335 0 0) 0x7f91ac130fe0 con 0x7f91ac122f40 2020-03-19T23:11:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 -- 172.21.15.38:0/2207013616 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac130cd0 con 0x7f91ac1224c0 2020-03-19T23:11:28.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> v2:172.21.15.38:3300/0 conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=READY pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.520 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f919c00f870 con 0x7f91ac1224c0 2020-03-19T23:11:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c007150 con 0x7f91ac1224c0 2020-03-19T23:11:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f919c0073f0 con 0x7f91ac1224c0 2020-03-19T23:11:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.488+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac1102f0 con 0x7f91ac1224c0 2020-03-19T23:11:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac111900 con 0x7f91ac1224c0 2020-03-19T23:11:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f919c0056c0 con 0x7f91ac1224c0 2020-03-19T23:11:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91a97fa700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 0x7f918c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91ac090b60 con 0x7f91ac1224c0 2020-03-19T23:11:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f919c034610 con 0x7f91ac1224c0 2020-03-19T23:11:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91ab7fe700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 0x7f918c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91abfff700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=THROTTLE_DONE pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91a97fa700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7f918c02dc40 0x7f918c030060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 --> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f918c0305f0 con 0x7f918c02dc40 2020-03-19T23:11:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.489+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f919c005e70 con 0x7f91ac1224c0 2020-03-19T23:11:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.490+0000 7f91ab7fe700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 0x7f918c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f91a0001260 tx=0x7f91a0001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.490+0000 7f91b0bb2700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7f918c02dc40 0x7f918c030060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.490+0000 7f91ab7fe700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 0x7f918c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f91a0001260 tx=0x7f91a0001740).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.490+0000 7f91ab7fe700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 0x7f918c02a740 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f91a0001260 tx=0x7f91a0001740).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.490+0000 7f91b0bb2700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7f918c02dc40 0x7f918c030060 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.491+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== osd.1 v2:172.21.15.38:6800/20730 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7f918c0305f0 con 0x7f918c02dc40 2020-03-19T23:11:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.507+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 --> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f91ac055d70 con 0x7f918c02dc40 2020-03-19T23:11:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91a97fa700 1 -- 172.21.15.38:0/2207013616 <== osd.1 v2:172.21.15.38:6800/20730 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f91ac055d70 con 0x7f918c02dc40 2020-03-19T23:11:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7f918c02dc40 msgr2=0x7f918c030060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7f918c02dc40 0x7f918c030060 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 msgr2=0x7f918c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 0x7f918c02a740 secure :-1 s=READY pgs=70 cs=0 l=1 rx=0x7f91a0001260 tx=0x7f91a0001740).stop 2020-03-19T23:11:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 msgr2=0x7f91ac1121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 secure :-1 s=READY pgs=281 cs=0 l=1 rx=0x7f919c002f30 tx=0x7f919c007a40).stop 2020-03-19T23:11:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 shutdown_connections 2020-03-19T23:11:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6800/20730,v1:172.21.15.38:6801/20730] conn(0x7f918c02dc40 0x7f918c030060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91ac1224c0 0x7f91ac1121f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 --2- 172.21.15.38:0/2207013616 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f918c028280 0x7f918c02a740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 >> 172.21.15.38:0/2207013616 conn(0x7f91ac091390 msgr2=0x7f91ac121990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 shutdown_connections 2020-03-19T23:11:28.526 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.508+0000 7f91b2615700 1 -- 172.21.15.38:0/2207013616 wait complete. 2020-03-19T23:11:28.526 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2020-03-19T23:11:28.527 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-03-19T23:11:28.527 INFO:teuthology.orchestra.run.smithi038.stdout:} 2020-03-19T23:11:28.528 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 1 2020-03-19T23:11:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.639+0000 7f0ca3b00700 1 Processor -- start 2020-03-19T23:11:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.639+0000 7f0ca3b00700 1 -- start start 2020-03-19T23:11:28.646 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca3b00700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0663e0 0x7f0c9c127280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:28.646 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca3b00700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c0604a0 con 0x7f0c9c0658f0 2020-03-19T23:11:28.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca3b00700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9c060810 con 0x7f0c9c0663e0 2020-03-19T23:11:28.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0663e0 0x7f0c9c127280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca189c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0c9c0658f0 0x7f0c9c065cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44336/0 (socket says 172.21.15.38:44336) 2020-03-19T23:11:28.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca189c700 1 -- 172.21.15.38:0/2150820467 learned_addr learned my addr 172.21.15.38:0/2150820467 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 -- 172.21.15.38:0/2150820467 >> v1:172.21.15.38:6789/0 conn(0x7f0c9c0658f0 legacy=0x7f0c9c065cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca089a700 1 -- 172.21.15.38:0/2150820467 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601381911 0 0) 0x7f0c9c0604a0 con 0x7f0c9c0658f0 2020-03-19T23:11:28.647 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 -- 172.21.15.38:0/2150820467 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c060110 con 0x7f0c9c0663e0 2020-03-19T23:11:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 --2- 172.21.15.38:0/2150820467 >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0663e0 0x7f0c9c127280 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f0c8c002f30 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 --2- 172.21.15.38:0/2150820467 >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0663e0 0x7f0c9c127280 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f0c8c002f30 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 --2- 172.21.15.38:0/2150820467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0663e0 0x7f0c9c127280 secure :-1 s=READY pgs=283 cs=0 l=1 rx=0x7f0c8c002f30 tx=0x7f0c8c007a40).ready entity=mon.0 client_cookie=2b5c5fafcdab2701 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 --2- 172.21.15.38:0/2150820467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0663e0 0x7f0c9c127280 secure :-1 s=THROTTLE_DONE pgs=283 cs=0 l=1 rx=0x7f0c8c002f30 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca089a700 1 -- 172.21.15.38:0/2150820467 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0c8c008e80 con 0x7f0c9c0663e0 2020-03-19T23:11:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.640+0000 7f0ca109b700 1 --2- 172.21.15.38:0/2150820467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0663e0 0x7f0c9c127280 secure :-1 s=THROTTLE_DONE pgs=283 cs=0 l=1 rx=0x7f0c8c002f30 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca109b700 1 --2- 172.21.15.38:0/2150820467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0663e0 0x7f0c9c127280 secure :-1 s=THROTTLE_DONE pgs=283 cs=0 l=1 rx=0x7f0c8c002f30 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca089a700 1 -- 172.21.15.38:0/2150820467 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c0113f0 con 0x7f0c9c0663e0 2020-03-19T23:11:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca089a700 1 -- 172.21.15.38:0/2150820467 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0c8c0116d0 con 0x7f0c9c0663e0 2020-03-19T23:11:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca3b00700 1 -- 172.21.15.38:0/2150820467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0663e0 msgr2=0x7f0c9c127280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca3b00700 1 --2- 172.21.15.38:0/2150820467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0663e0 0x7f0c9c127280 secure :-1 s=READY pgs=283 cs=0 l=1 rx=0x7f0c8c002f30 tx=0x7f0c8c007a40).stop 2020-03-19T23:11:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca3b00700 1 -- 172.21.15.38:0/2150820467 shutdown_connections 2020-03-19T23:11:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca3b00700 1 --2- 172.21.15.38:0/2150820467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0663e0 0x7f0c9c127280 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca3b00700 1 -- 172.21.15.38:0/2150820467 >> 172.21.15.38:0/2150820467 conn(0x7f0c9c091390 msgr2=0x7f0c9c0917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca3b00700 1 -- 172.21.15.38:0/2150820467 shutdown_connections 2020-03-19T23:11:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.641+0000 7f0ca3b00700 1 -- 172.21.15.38:0/2150820467 wait complete. 2020-03-19T23:11:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca3b00700 1 Processor -- start 2020-03-19T23:11:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca3b00700 1 -- start start 2020-03-19T23:11:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca3b00700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0658f0 0x7f0c9c113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca3b00700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c128bf0 con 0x7f0c9c0663e0 2020-03-19T23:11:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca3b00700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9c127f70 con 0x7f0c9c0658f0 2020-03-19T23:11:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca189c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0658f0 0x7f0c9c113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca189c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0658f0 0x7f0c9c113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32968/0 (socket says 172.21.15.38:32968) 2020-03-19T23:11:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.642+0000 7f0ca189c700 1 -- 172.21.15.38:0/800436568 learned_addr learned my addr 172.21.15.38:0/800436568 (peer_addr_for_me v2:172.21.15.38:0/0) 2020-03-19T23:11:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 -- 172.21.15.38:0/800436568 >> v1:172.21.15.38:6789/0 conn(0x7f0c9c0663e0 legacy=0x7f0c9c130e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 -- 172.21.15.38:0/800436568 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c1137c0 con 0x7f0c9c0658f0 2020-03-19T23:11:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251721858 0 0) 0x7f0c9c128bf0 con 0x7f0c9c0663e0 2020-03-19T23:11:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> v2:172.21.15.38:3300/0 conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=READY pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0c94012370 con 0x7f0c9c0658f0 2020-03-19T23:11:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c940124d0 con 0x7f0c9c0658f0 2020-03-19T23:11:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c131580 con 0x7f0c9c0658f0 2020-03-19T23:11:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f0c94012770 con 0x7f0c9c0658f0 2020-03-19T23:11:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c111a40 con 0x7f0c9c0658f0 2020-03-19T23:11:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f0c940129e0 con 0x7f0c9c0658f0 2020-03-19T23:11:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.643+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.644+0000 7f0c8affd700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.644+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f0c94034900 con 0x7f0c9c0658f0 2020-03-19T23:11:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.644+0000 7f0ca109b700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.644+0000 7f0ca109b700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f0c8c002ed0 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.644+0000 7f0ca109b700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f0c8c002ed0 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.644+0000 7f0ca109b700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7f0c8c002ed0 tx=0x7f0c8c007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.644+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c056db0 con 0x7f0c9c0658f0 2020-03-19T23:11:28.655 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.647+0000 7f0ca189c700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=THROTTLE_DONE pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.655 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.647+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f0c94011af0 con 0x7f0c9c0658f0 2020-03-19T23:11:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.794+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c9c0693f0 con 0x7f0c7c028280 2020-03-19T23:11:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.795+0000 7f0ca109b700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 secure :-1 s=THROTTLE_DONE pgs=71 cs=0 l=1 rx=0x7f0c8c002ed0 tx=0x7f0c8c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.797 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.795+0000 7f0c8affd700 1 -- 172.21.15.38:0/800436568 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0c9c0693f0 con 0x7f0c7c028280 2020-03-19T23:11:28.798 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 msgr2=0x7f0c7c02a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 secure :-1 s=READY pgs=71 cs=0 l=1 rx=0x7f0c8c002ed0 tx=0x7f0c8c007a40).stop 2020-03-19T23:11:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 msgr2=0x7f0c9c113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 secure :-1 s=READY pgs=285 cs=0 l=1 rx=0x7f0c9c112fe0 tx=0x7f0c94010d10).stop 2020-03-19T23:11:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 shutdown_connections 2020-03-19T23:11:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c9c0658f0 0x7f0c9c113160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 --2- 172.21.15.38:0/800436568 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f0c7c028280 0x7f0c7c02a740 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.799 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 >> 172.21.15.38:0/800436568 conn(0x7f0c9c091390 msgr2=0x7f0c9c08e060 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 shutdown_connections 2020-03-19T23:11:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.796+0000 7f0ca3b00700 1 -- 172.21.15.38:0/800436568 wait complete. 2020-03-19T23:11:28.800 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 1 to deep-scrub 2020-03-19T23:11:28.808 INFO:tasks.ceph:Scrubbing osd.2 2020-03-19T23:11:28.808 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 config set osd_debug_deep_scrub_sleep 0 2020-03-19T23:11:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.924+0000 7f7c5558e700 1 Processor -- start 2020-03-19T23:11:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.924+0000 7f7c5558e700 1 -- start start 2020-03-19T23:11:28.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.924+0000 7f7c5558e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c50125060 0x7f7c50127450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:28.928 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.924+0000 7f7c5558e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c50060f10 con 0x7f7c50065970 2020-03-19T23:11:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.924+0000 7f7c5558e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c50061490 con 0x7f7c50125060 2020-03-19T23:11:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c50125060 0x7f7c50127450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4effd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7c50065970 0x7f7c50065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44346/0 (socket says 172.21.15.38:44346) 2020-03-19T23:11:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4effd700 1 -- 172.21.15.38:0/1076861299 learned_addr learned my addr 172.21.15.38:0/1076861299 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 -- 172.21.15.38:0/1076861299 >> v1:172.21.15.38:6789/0 conn(0x7f7c50065970 legacy=0x7f7c50065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 -- 172.21.15.38:0/1076861299 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50061730 con 0x7f7c50125060 2020-03-19T23:11:28.929 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4dffb700 1 -- 172.21.15.38:0/1076861299 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988200491 0 0) 0x7f7c480010d0 con 0x7f7c50065970 2020-03-19T23:11:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/1076861299 >> v2:172.21.15.38:3300/0 conn(0x7f7c50125060 0x7f7c50127450 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f7c440032b0 tx=0x7f7c44007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/1076861299 >> v2:172.21.15.38:3300/0 conn(0x7f7c50125060 0x7f7c50127450 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f7c440032b0 tx=0x7f7c44007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/1076861299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50125060 0x7f7c50127450 secure :-1 s=READY pgs=287 cs=0 l=1 rx=0x7f7c440032b0 tx=0x7f7c44007dc0).ready entity=mon.0 client_cookie=fc4fdee5ce347a server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/1076861299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50125060 0x7f7c50127450 secure :-1 s=THROTTLE_DONE pgs=287 cs=0 l=1 rx=0x7f7c440032b0 tx=0x7f7c44007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4dffb700 1 -- 172.21.15.38:0/1076861299 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7c44011d70 con 0x7f7c50125060 2020-03-19T23:11:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/1076861299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50125060 0x7f7c50127450 secure :-1 s=THROTTLE_DONE pgs=287 cs=0 l=1 rx=0x7f7c440032b0 tx=0x7f7c44007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.930 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/1076861299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50125060 0x7f7c50127450 secure :-1 s=THROTTLE_DONE pgs=287 cs=0 l=1 rx=0x7f7c440032b0 tx=0x7f7c44007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4dffb700 1 -- 172.21.15.38:0/1076861299 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4400a160 con 0x7f7c50125060 2020-03-19T23:11:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c4dffb700 1 -- 172.21.15.38:0/1076861299 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7c4400a310 con 0x7f7c50125060 2020-03-19T23:11:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c5558e700 1 -- 172.21.15.38:0/1076861299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50125060 msgr2=0x7f7c50127450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.925+0000 7f7c5558e700 1 --2- 172.21.15.38:0/1076861299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50125060 0x7f7c50127450 secure :-1 s=READY pgs=287 cs=0 l=1 rx=0x7f7c440032b0 tx=0x7f7c44007dc0).stop 2020-03-19T23:11:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.926+0000 7f7c5558e700 1 -- 172.21.15.38:0/1076861299 shutdown_connections 2020-03-19T23:11:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.926+0000 7f7c5558e700 1 --2- 172.21.15.38:0/1076861299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50125060 0x7f7c50127450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.931 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.926+0000 7f7c5558e700 1 -- 172.21.15.38:0/1076861299 >> 172.21.15.38:0/1076861299 conn(0x7f7c50091390 msgr2=0x7f7c50091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.926+0000 7f7c5558e700 1 -- 172.21.15.38:0/1076861299 shutdown_connections 2020-03-19T23:11:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.926+0000 7f7c5558e700 1 -- 172.21.15.38:0/1076861299 wait complete. 2020-03-19T23:11:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.926+0000 7f7c5558e700 1 Processor -- start 2020-03-19T23:11:28.932 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c5558e700 1 -- start start 2020-03-19T23:11:28.935 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c5558e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c50065970 0x7f7c50130b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.935 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c5558e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c501308b0 con 0x7f7c50125060 2020-03-19T23:11:28.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c5558e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c50112ca0 con 0x7f7c50065970 2020-03-19T23:11:28.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4effd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c50065970 0x7f7c50130b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4e7fc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7c50125060 0x7f7c50130fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44350/0 (socket says 172.21.15.38:44350) 2020-03-19T23:11:28.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4e7fc700 1 -- 172.21.15.38:0/454799464 learned_addr learned my addr 172.21.15.38:0/454799464 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:28.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334933414 0 0) 0x7f7c501308b0 con 0x7f7c50125060 2020-03-19T23:11:28.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c2c002fe0 con 0x7f7c50125060 2020-03-19T23:11:28.936 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4effd700 1 -- 172.21.15.38:0/454799464 >> v1:172.21.15.38:6789/0 conn(0x7f7c50125060 legacy=0x7f7c50130fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4effd700 1 -- 172.21.15.38:0/454799464 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50111480 con 0x7f7c50065970 2020-03-19T23:11:28.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> v2:172.21.15.38:3300/0 conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> v2:172.21.15.38:3300/0 conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.927+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=READY pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.937 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7c48005b20 con 0x7f7c50065970 2020-03-19T23:11:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c48005c80 con 0x7f7c50065970 2020-03-19T23:11:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c501316c0 con 0x7f7c50065970 2020-03-19T23:11:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f7c480060c0 con 0x7f7c50065970 2020-03-19T23:11:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5010fa10 con 0x7f7c50065970 2020-03-19T23:11:28.938 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f7c48006330 con 0x7f7c50065970 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c3ffff700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 0x7f7c2c02ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c5008fd20 con 0x7f7c50065970 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.928+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f7c48033ef0 con 0x7f7c50065970 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 0x7f7c2c02ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c3ffff700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7c2c02e080 0x7f7c2c0304a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:28.939 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 --> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c2c030a30 con 0x7f7c2c02e080 2020-03-19T23:11:28.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c4effd700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=THROTTLE_DONE pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c480090b0 con 0x7f7c50065970 2020-03-19T23:11:28.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 0x7f7c2c02ac80 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f7c44000f00 tx=0x7f7c44007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c4f7fe700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7c2c02e080 0x7f7c2c0304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:28.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 0x7f7c2c02ac80 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f7c44000f00 tx=0x7f7c44007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:28.940 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c4e7fc700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 0x7f7c2c02ac80 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7f7c44000f00 tx=0x7f7c44007dc0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.941 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.929+0000 7f7c4f7fe700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7c2c02e080 0x7f7c2c0304a0 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:28.941 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.930+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== osd.2 v2:172.21.15.38:6816/20739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23288 (crc 0 0 0) 0x7f7c2c030a30 con 0x7f7c2c02e080 2020-03-19T23:11:28.949 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.947+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 --> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7c50094660 con 0x7f7c2c02e080 2020-03-19T23:11:28.950 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.947+0000 7f7c3ffff700 1 -- 172.21.15.38:0/454799464 <== osd.2 v2:172.21.15.38:6816/20739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7c50094660 con 0x7f7c2c02e080 2020-03-19T23:11:28.950 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7c2c02e080 msgr2=0x7f7c2c0304a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.951 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7c2c02e080 0x7f7c2c0304a0 crc :-1 s=READY pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.951 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 msgr2=0x7f7c2c02ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.951 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 0x7f7c2c02ac80 secure :-1 s=READY pgs=72 cs=0 l=1 rx=0x7f7c44000f00 tx=0x7f7c44007dc0).stop 2020-03-19T23:11:28.951 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 msgr2=0x7f7c50130b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:28.951 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 secure :-1 s=READY pgs=289 cs=0 l=1 rx=0x7f7c50111800 tx=0x7f7c48010860).stop 2020-03-19T23:11:28.951 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 shutdown_connections 2020-03-19T23:11:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c50065970 0x7f7c50130b50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6816/20739,v1:172.21.15.38:6817/20739] conn(0x7f7c2c02e080 0x7f7c2c0304a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 --2- 172.21.15.38:0/454799464 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f7c2c0287c0 0x7f7c2c02ac80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 >> 172.21.15.38:0/454799464 conn(0x7f7c50091390 msgr2=0x7f7c5008dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 shutdown_connections 2020-03-19T23:11:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:28.948+0000 7f7c5558e700 1 -- 172.21.15.38:0/454799464 wait complete. 2020-03-19T23:11:28.952 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2020-03-19T23:11:28.953 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2020-03-19T23:11:28.953 INFO:teuthology.orchestra.run.smithi038.stdout:} 2020-03-19T23:11:28.960 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd deep-scrub 2 2020-03-19T23:11:29.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f472ccb8700 1 Processor -- start 2020-03-19T23:11:29.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f472ccb8700 1 -- start start 2020-03-19T23:11:29.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f472ccb8700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4728126ae0 0x7f4728128ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:29.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f472ccb8700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4728090830 con 0x7f4728062550 2020-03-19T23:11:29.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f472ccb8700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4728120690 con 0x7f4728126ae0 2020-03-19T23:11:29.076 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f4726ffd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4728126ae0 0x7f4728128ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:29.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f47277fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4728062550 0x7f47281209a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44358/0 (socket says 172.21.15.38:44358) 2020-03-19T23:11:29.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.072+0000 7f47277fe700 1 -- 172.21.15.38:0/3318072649 learned_addr learned my addr 172.21.15.38:0/3318072649 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:29.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 -- 172.21.15.38:0/3318072649 >> v1:172.21.15.38:6789/0 conn(0x7f4728062550 legacy=0x7f47281209a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 -- 172.21.15.38:0/3318072649 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728090d00 con 0x7f4728126ae0 2020-03-19T23:11:29.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f47267fc700 1 -- 172.21.15.38:0/3318072649 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401101452 0 0) 0x7f4728090830 con 0x7f4728062550 2020-03-19T23:11:29.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 --2- 172.21.15.38:0/3318072649 >> v2:172.21.15.38:3300/0 conn(0x7f4728126ae0 0x7f4728128ed0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f471c002f30 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.077 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 --2- 172.21.15.38:0/3318072649 >> v2:172.21.15.38:3300/0 conn(0x7f4728126ae0 0x7f4728128ed0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f471c002f30 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 --2- 172.21.15.38:0/3318072649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728126ae0 0x7f4728128ed0 secure :-1 s=READY pgs=291 cs=0 l=1 rx=0x7f471c002f30 tx=0x7f471c007a40).ready entity=mon.0 client_cookie=119e4ea71a0284ff server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:29.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 --2- 172.21.15.38:0/3318072649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728126ae0 0x7f4728128ed0 secure :-1 s=THROTTLE_DONE pgs=291 cs=0 l=1 rx=0x7f471c002f30 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f47267fc700 1 -- 172.21.15.38:0/3318072649 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f471c008e80 con 0x7f4728126ae0 2020-03-19T23:11:29.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 --2- 172.21.15.38:0/3318072649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728126ae0 0x7f4728128ed0 secure :-1 s=THROTTLE_DONE pgs=291 cs=0 l=1 rx=0x7f471c002f30 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f47267fc700 1 -- 172.21.15.38:0/3318072649 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c0113f0 con 0x7f4728126ae0 2020-03-19T23:11:29.078 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f4726ffd700 1 --2- 172.21.15.38:0/3318072649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728126ae0 0x7f4728128ed0 secure :-1 s=THROTTLE_DONE pgs=291 cs=0 l=1 rx=0x7f471c002f30 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f47267fc700 1 -- 172.21.15.38:0/3318072649 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f471c0116d0 con 0x7f4728126ae0 2020-03-19T23:11:29.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f472ccb8700 1 -- 172.21.15.38:0/3318072649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728126ae0 msgr2=0x7f4728128ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f472ccb8700 1 --2- 172.21.15.38:0/3318072649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728126ae0 0x7f4728128ed0 secure :-1 s=READY pgs=291 cs=0 l=1 rx=0x7f471c002f30 tx=0x7f471c007a40).stop 2020-03-19T23:11:29.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f472ccb8700 1 -- 172.21.15.38:0/3318072649 shutdown_connections 2020-03-19T23:11:29.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f472ccb8700 1 --2- 172.21.15.38:0/3318072649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728126ae0 0x7f4728128ed0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:29.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f472ccb8700 1 -- 172.21.15.38:0/3318072649 >> 172.21.15.38:0/3318072649 conn(0x7f4728091390 msgr2=0x7f4728091770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:29.079 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f472ccb8700 1 -- 172.21.15.38:0/3318072649 shutdown_connections 2020-03-19T23:11:29.080 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.073+0000 7f472ccb8700 1 -- 172.21.15.38:0/3318072649 wait complete. 2020-03-19T23:11:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.074+0000 7f472ccb8700 1 Processor -- start 2020-03-19T23:11:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.074+0000 7f472ccb8700 1 -- start start 2020-03-19T23:11:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f472ccb8700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4728062550 0x7f472810c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f472ccb8700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472811f860 con 0x7f4728126ae0 2020-03-19T23:11:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f472ccb8700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472811fec0 con 0x7f4728062550 2020-03-19T23:11:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4728062550 0x7f472810c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f4726ffd700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4728126ae0 0x7f4728065660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44364/0 (socket says 172.21.15.38:44364) 2020-03-19T23:11:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f4726ffd700 1 -- 172.21.15.38:0/1007742588 learned_addr learned my addr 172.21.15.38:0/1007742588 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 -- 172.21.15.38:0/1007742588 >> v1:172.21.15.38:6789/0 conn(0x7f4728126ae0 legacy=0x7f4728065660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205566140 0 0) 0x7f472811f860 con 0x7f4728126ae0 2020-03-19T23:11:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 -- 172.21.15.38:0/1007742588 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472810c960 con 0x7f4728062550 2020-03-19T23:11:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> v2:172.21.15.38:3300/0 conn(0x7f4728062550 0x7f472810c500 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> v2:172.21.15.38:3300/0 conn(0x7f4728062550 0x7f472810c500 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=READY pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f4718005b20 con 0x7f4728062550 2020-03-19T23:11:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718005c80 con 0x7f4728062550 2020-03-19T23:11:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47280667a0 con 0x7f4728062550 2020-03-19T23:11:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f47180060c0 con 0x7f4728062550 2020-03-19T23:11:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.075+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472810cbc0 con 0x7f4728062550 2020-03-19T23:11:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.076+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.076+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.076+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f471800a320 con 0x7f4728062550 2020-03-19T23:11:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.076+0000 7f4724ff9700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.076+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f4718034000 con 0x7f4728062550 2020-03-19T23:11:29.086 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.076+0000 7f4726ffd700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.077+0000 7f4726ffd700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f471c0101c0 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.077+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4728056db0 con 0x7f4728062550 2020-03-19T23:11:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.077+0000 7f4726ffd700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f471c0101c0 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.077+0000 7f4726ffd700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f471c0101c0 tx=0x7f471c007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.080+0000 7f47277fe700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=THROTTLE_DONE pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.087 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.080+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f4718006220 con 0x7f4728062550 2020-03-19T23:11:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.229+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f472806aaf0 con 0x7f4708028250 2020-03-19T23:11:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.230+0000 7f4726ffd700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 secure :-1 s=THROTTLE_DONE pgs=73 cs=0 l=1 rx=0x7f471c0101c0 tx=0x7f471c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.230+0000 7f4724ff9700 1 -- 172.21.15.38:0/1007742588 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f472806aaf0 con 0x7f4708028250 2020-03-19T23:11:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 msgr2=0x7f470802a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 secure :-1 s=READY pgs=73 cs=0 l=1 rx=0x7f471c0101c0 tx=0x7f471c007a40).stop 2020-03-19T23:11:29.233 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 msgr2=0x7f472810c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 secure :-1 s=READY pgs=293 cs=0 l=1 rx=0x7f4718000b80 tx=0x7f4718010930).stop 2020-03-19T23:11:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 shutdown_connections 2020-03-19T23:11:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4728062550 0x7f472810c500 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 --2- 172.21.15.38:0/1007742588 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f4708028250 0x7f470802a710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 >> 172.21.15.38:0/1007742588 conn(0x7f4728091390 msgr2=0x7f4728124be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 shutdown_connections 2020-03-19T23:11:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.231+0000 7f472ccb8700 1 -- 172.21.15.38:0/1007742588 wait complete. 2020-03-19T23:11:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 2 to deep-scrub 2020-03-19T23:11:29.243 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T23:11:29.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa38c7ce700 1 Processor -- start 2020-03-19T23:11:29.357 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa38c7ce700 1 -- start start 2020-03-19T23:11:29.361 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa38c7ce700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa384127640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:29.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa38c7ce700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38408eee0 con 0x7fa3841221d0 2020-03-19T23:11:29.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa38c7ce700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa384091060 con 0x7fa384125250 2020-03-19T23:11:29.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa389d69700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa384127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:29.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa38a56a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa3841221d0 0x7fa3841225b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44368/0 (socket says 172.21.15.38:44368) 2020-03-19T23:11:29.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.355+0000 7fa38a56a700 1 -- 172.21.15.38:0/3355777162 learned_addr learned my addr 172.21.15.38:0/3355777162 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:29.362 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 -- 172.21.15.38:0/3355777162 >> v1:172.21.15.38:6789/0 conn(0x7fa3841221d0 legacy=0x7fa3841225b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 -- 172.21.15.38:0/3355777162 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa384121fd0 con 0x7fa384125250 2020-03-19T23:11:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 --2- 172.21.15.38:0/3355777162 >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa384127640 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7fa3800032b0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 --2- 172.21.15.38:0/3355777162 >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa384127640 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7fa3800032b0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 --2- 172.21.15.38:0/3355777162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa384127640 secure :-1 s=READY pgs=295 cs=0 l=1 rx=0x7fa3800032b0 tx=0x7fa380007dc0).ready entity=mon.0 client_cookie=8ba3520cf03477f server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 --2- 172.21.15.38:0/3355777162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa384127640 secure :-1 s=THROTTLE_DONE pgs=295 cs=0 l=1 rx=0x7fa3800032b0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389568700 1 -- 172.21.15.38:0/3355777162 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa380011d70 con 0x7fa384125250 2020-03-19T23:11:29.363 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 --2- 172.21.15.38:0/3355777162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa384127640 secure :-1 s=THROTTLE_DONE pgs=295 cs=0 l=1 rx=0x7fa3800032b0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389d69700 1 --2- 172.21.15.38:0/3355777162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa384127640 secure :-1 s=THROTTLE_DONE pgs=295 cs=0 l=1 rx=0x7fa3800032b0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389568700 1 -- 172.21.15.38:0/3355777162 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38000a160 con 0x7fa384125250 2020-03-19T23:11:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa389568700 1 -- 172.21.15.38:0/3355777162 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa38000a350 con 0x7fa384125250 2020-03-19T23:11:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa38c7ce700 1 -- 172.21.15.38:0/3355777162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 msgr2=0x7fa384127640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa38c7ce700 1 --2- 172.21.15.38:0/3355777162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa384127640 secure :-1 s=READY pgs=295 cs=0 l=1 rx=0x7fa3800032b0 tx=0x7fa380007dc0).stop 2020-03-19T23:11:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa38c7ce700 1 -- 172.21.15.38:0/3355777162 shutdown_connections 2020-03-19T23:11:29.364 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa38c7ce700 1 --2- 172.21.15.38:0/3355777162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa384127640 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa38c7ce700 1 -- 172.21.15.38:0/3355777162 >> 172.21.15.38:0/3355777162 conn(0x7fa384091540 msgr2=0x7fa384091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa38c7ce700 1 -- 172.21.15.38:0/3355777162 shutdown_connections 2020-03-19T23:11:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.356+0000 7fa38c7ce700 1 -- 172.21.15.38:0/3355777162 wait complete. 2020-03-19T23:11:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.357+0000 7fa38c7ce700 1 Processor -- start 2020-03-19T23:11:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.357+0000 7fa38c7ce700 1 -- start start 2020-03-19T23:11:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa38c7ce700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa38411cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:29.365 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa38c7ce700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38411f470 con 0x7fa3841221d0 2020-03-19T23:11:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa38c7ce700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3841329c0 con 0x7fa384125250 2020-03-19T23:11:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa38a56a700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa3841221d0 0x7fa384132cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44372/0 (socket says 172.21.15.38:44372) 2020-03-19T23:11:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa38411cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa38a56a700 1 -- 172.21.15.38:0/2949921809 learned_addr learned my addr 172.21.15.38:0/2949921809 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 -- 172.21.15.38:0/2949921809 >> v1:172.21.15.38:6789/0 conn(0x7fa3841221d0 legacy=0x7fa384132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469682588 0 0) 0x7fa38411f470 con 0x7fa3841221d0 2020-03-19T23:11:29.366 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 -- 172.21.15.38:0/2949921809 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38411eb20 con 0x7fa384125250 2020-03-19T23:11:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> v2:172.21.15.38:3300/0 conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=READY pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa380005a40 con 0x7fa384125250 2020-03-19T23:11:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.367 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa380005ba0 con 0x7fa384125250 2020-03-19T23:11:29.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa384064da0 con 0x7fa384125250 2020-03-19T23:11:29.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7fa380005e40 con 0x7fa384125250 2020-03-19T23:11:29.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.358+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa384133620 con 0x7fa384125250 2020-03-19T23:11:29.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.359+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.359+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7fa38000f570 con 0x7fa384125250 2020-03-19T23:11:29.368 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.359+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.359+0000 7fa3777fe700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.359+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7fa380035260 con 0x7fa384125250 2020-03-19T23:11:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.359+0000 7fa38a56a700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.359+0000 7fa38a56a700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7fa378000ef0 tx=0x7fa378001680).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.360+0000 7fa38a56a700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7fa378000ef0 tx=0x7fa378001680).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.360+0000 7fa38a56a700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7fa378000ef0 tx=0x7fa378001680).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:29.369 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.360+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa384132b30 con 0x7fa384125250 2020-03-19T23:11:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.363+0000 7fa389d69700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=THROTTLE_DONE pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.370 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.363+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7fa384132b30 con 0x7fa384125250 2020-03-19T23:11:29.509 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.507+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa384065bb0 con 0x7fa364028200 2020-03-19T23:11:29.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.509+0000 7fa38a56a700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 secure :-1 s=THROTTLE_DONE pgs=74 cs=0 l=1 rx=0x7fa378000ef0 tx=0x7fa378001680).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:29.512 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.509+0000 7fa3777fe700 1 -- 172.21.15.38:0/2949921809 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+74039 (secure 0 0 0) 0x7fa37800c270 con 0x7fa364028200 2020-03-19T23:11:29.512 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:29.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 msgr2=0x7fa36402a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.513 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 secure :-1 s=READY pgs=74 cs=0 l=1 rx=0x7fa378000ef0 tx=0x7fa378001680).stop 2020-03-19T23:11:29.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 msgr2=0x7fa38411cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:29.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 secure :-1 s=READY pgs=297 cs=0 l=1 rx=0x7fa3840655c0 tx=0x7fa380007dc0).stop 2020-03-19T23:11:29.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 shutdown_connections 2020-03-19T23:11:29.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7fa364028200 0x7fa36402a6c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:29.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 --2- 172.21.15.38:0/2949921809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa384125250 0x7fa38411cfd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:29.514 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 >> 172.21.15.38:0/2949921809 conn(0x7fa384091540 msgr2=0x7fa38408ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:29.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 shutdown_connections 2020-03-19T23:11:29.515 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:29.511+0000 7fa38c7ce700 1 -- 172.21.15.38:0/2949921809 wait complete. 2020-03-19T23:11:29.515 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2020-03-19T23:11:29.523 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2020-03-19T23:11:27.618999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146416,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426384,"statfs":{"total":322122547200,"available":318900617216,"internally_reserved":3221225472,"allocated":679936,"data_stored":238689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":432,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.249636"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.645982+0000","last_change":"2020-03-19T23:11:16.354394+0000","last_active":"2020-03-19T23:11:20.645982+0000","last_peered":"2020-03-19T23:11:20.645982+0000","last_clean":"2020-03-19T23:11:20.645982+0000","last_became_active":"2020-03-19T23:11:16.353972+0000","last_became_peered":"2020-03-19T23:11:16.353972+0000","last_unstale":"2020-03-19T23:11:20.645982+0000","last_undegraded":"2020-03-19T23:11:20.645982+0000","last_fullsized":"2020-03-19T23:11:20.645982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370689+0000","last_change":"2020-03-19T23:11:04.653879+0000","last_active":"2020-03-19T23:11:20.370689+0000","last_peered":"2020-03-19T23:11:20.370689+0000","last_clean":"2020-03-19T23:11:20.370689+0000","last_became_active":"2020-03-19T23:11:04.653521+0000","last_became_peered":"2020-03-19T23:11:04.653521+0000","last_unstale":"2020-03-19T23:11:20.370689+0000","last_undegraded":"2020-03-19T23:11:20.370689+0000","last_fullsized":"2020-03-19T23:11:20.370689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370827+0000","last_change":"2020-03-19T23:11:01.250472+0000","last_active":"2020-03-19T23:11:20.370827+0000","last_peered":"2020-03-19T23:11:20.370827+0000","last_clean":"2020-03-19T23:11:20.370827+0000","last_became_active":"2020-03-19T23:11:01.250084+0000","last_became_peered":"2020-03-19T23:11:01.250084+0000","last_unstale":"2020-03-19T23:11:20.370827+0000","last_undegraded":"2020-03-19T23:11:20.370827+0000","last_fullsized":"2020-03-19T23:11:20.370827+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'4","reported_seq":"21","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378847+0000","last_change":"2020-03-19T23:11:04.285615+0000","last_active":"2020-03-19T23:11:21.378847+0000","last_peered":"2020-03-19T23:11:21.378847+0000","last_clean":"2020-03-19T23:11:21.378847+0000","last_became_active":"2020-03-19T23:11:04.284829+0000","last_became_peered":"2020-03-19T23:11:04.284829+0000","last_unstale":"2020-03-19T23:11:21.378847+0000","last_undegraded":"2020-03-19T23:11:21.378847+0000","last_fullsized":"2020-03-19T23:11:21.378847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370486+0000","last_change":"2020-03-19T23:11:16.355229+0000","last_active":"2020-03-19T23:11:20.370486+0000","last_peered":"2020-03-19T23:11:20.370486+0000","last_clean":"2020-03-19T23:11:20.370486+0000","last_became_active":"2020-03-19T23:11:16.354846+0000","last_became_peered":"2020-03-19T23:11:16.354846+0000","last_unstale":"2020-03-19T23:11:20.370486+0000","last_undegraded":"2020-03-19T23:11:20.370486+0000","last_fullsized":"2020-03-19T23:11:20.370486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.646194+0000","last_change":"2020-03-19T23:11:04.654105+0000","last_active":"2020-03-19T23:11:20.646194+0000","last_peered":"2020-03-19T23:11:20.646194+0000","last_clean":"2020-03-19T23:11:20.646194+0000","last_became_active":"2020-03-19T23:11:04.653741+0000","last_became_peered":"2020-03-19T23:11:04.653741+0000","last_unstale":"2020-03-19T23:11:20.646194+0000","last_undegraded":"2020-03-19T23:11:20.646194+0000","last_fullsized":"2020-03-19T23:11:20.646194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378568+0000","last_change":"2020-03-19T23:11:01.251743+0000","last_active":"2020-03-19T23:11:21.378568+0000","last_peered":"2020-03-19T23:11:21.378568+0000","last_clean":"2020-03-19T23:11:21.378568+0000","last_became_active":"2020-03-19T23:11:01.251378+0000","last_became_peered":"2020-03-19T23:11:01.251378+0000","last_unstale":"2020-03-19T23:11:21.378568+0000","last_undegraded":"2020-03-19T23:11:21.378568+0000","last_fullsized":"2020-03-19T23:11:21.378568+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"24","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378816+0000","last_change":"2020-03-19T23:11:04.285389+0000","last_active":"2020-03-19T23:11:21.378816+0000","last_peered":"2020-03-19T23:11:21.378816+0000","last_clean":"2020-03-19T23:11:21.378816+0000","last_became_active":"2020-03-19T23:11:04.284979+0000","last_became_peered":"2020-03-19T23:11:04.284979+0000","last_unstale":"2020-03-19T23:11:21.378816+0000","last_undegraded":"2020-03-19T23:11:21.378816+0000","last_fullsized":"2020-03-19T23:11:21.378816+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.646141+0000","last_change":"2020-03-19T23:11:04.650535+0000","last_active":"2020-03-19T23:11:20.646141+0000","last_peered":"2020-03-19T23:11:20.646141+0000","last_clean":"2020-03-19T23:11:20.646141+0000","last_became_active":"2020-03-19T23:11:04.649316+0000","last_became_peered":"2020-03-19T23:11:04.649316+0000","last_unstale":"2020-03-19T23:11:20.646141+0000","last_undegraded":"2020-03-19T23:11:20.646141+0000","last_fullsized":"2020-03-19T23:11:20.646141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378502+0000","last_change":"2020-03-19T23:11:01.248950+0000","last_active":"2020-03-19T23:11:21.378502+0000","last_peered":"2020-03-19T23:11:21.378502+0000","last_clean":"2020-03-19T23:11:21.378502+0000","last_became_active":"2020-03-19T23:11:01.248513+0000","last_became_peered":"2020-03-19T23:11:01.248513+0000","last_unstale":"2020-03-19T23:11:21.378502+0000","last_undegraded":"2020-03-19T23:11:21.378502+0000","last_fullsized":"2020-03-19T23:11:21.378502+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'8","reported_seq":"24","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370532+0000","last_change":"2020-03-19T23:11:04.288163+0000","last_active":"2020-03-19T23:11:20.370532+0000","last_peered":"2020-03-19T23:11:20.370532+0000","last_clean":"2020-03-19T23:11:20.370532+0000","last_became_active":"2020-03-19T23:11:04.287139+0000","last_became_peered":"2020-03-19T23:11:04.287139+0000","last_unstale":"2020-03-19T23:11:20.370532+0000","last_undegraded":"2020-03-19T23:11:20.370532+0000","last_fullsized":"2020-03-19T23:11:20.370532+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378498+0000","last_change":"2020-03-19T23:11:04.654111+0000","last_active":"2020-03-19T23:11:21.378498+0000","last_peered":"2020-03-19T23:11:21.378498+0000","last_clean":"2020-03-19T23:11:21.378498+0000","last_became_active":"2020-03-19T23:11:04.653768+0000","last_became_peered":"2020-03-19T23:11:04.653768+0000","last_unstale":"2020-03-19T23:11:21.378498+0000","last_undegraded":"2020-03-19T23:11:21.378498+0000","last_fullsized":"2020-03-19T23:11:21.378498+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.646131+0000","last_change":"2020-03-19T23:11:01.252367+0000","last_active":"2020-03-19T23:11:20.646131+0000","last_peered":"2020-03-19T23:11:20.646131+0000","last_clean":"2020-03-19T23:11:20.646131+0000","last_became_active":"2020-03-19T23:11:01.251474+0000","last_became_peered":"2020-03-19T23:11:01.251474+0000","last_unstale":"2020-03-19T23:11:20.646131+0000","last_undegraded":"2020-03-19T23:11:20.646131+0000","last_fullsized":"2020-03-19T23:11:20.646131+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370670+0000","last_change":"2020-03-19T23:11:04.287593+0000","last_active":"2020-03-19T23:11:20.370670+0000","last_peered":"2020-03-19T23:11:20.370670+0000","last_clean":"2020-03-19T23:11:20.370670+0000","last_became_active":"2020-03-19T23:11:04.286797+0000","last_became_peered":"2020-03-19T23:11:04.286797+0000","last_unstale":"2020-03-19T23:11:20.370670+0000","last_undegraded":"2020-03-19T23:11:20.370670+0000","last_fullsized":"2020-03-19T23:11:20.370670+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378901+0000","last_change":"2020-03-19T23:11:04.650882+0000","last_active":"2020-03-19T23:11:21.378901+0000","last_peered":"2020-03-19T23:11:21.378901+0000","last_clean":"2020-03-19T23:11:21.378901+0000","last_became_active":"2020-03-19T23:11:04.650546+0000","last_became_peered":"2020-03-19T23:11:04.650546+0000","last_unstale":"2020-03-19T23:11:21.378901+0000","last_undegraded":"2020-03-19T23:11:21.378901+0000","last_fullsized":"2020-03-19T23:11:21.378901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378375+0000","last_change":"2020-03-19T23:11:01.249741+0000","last_active":"2020-03-19T23:11:21.378375+0000","last_peered":"2020-03-19T23:11:21.378375+0000","last_clean":"2020-03-19T23:11:21.378375+0000","last_became_active":"2020-03-19T23:11:01.249356+0000","last_became_peered":"2020-03-19T23:11:01.249356+0000","last_unstale":"2020-03-19T23:11:21.378375+0000","last_undegraded":"2020-03-19T23:11:21.378375+0000","last_fullsized":"2020-03-19T23:11:21.378375+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'5","reported_seq":"22","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378392+0000","last_change":"2020-03-19T23:11:04.284232+0000","last_active":"2020-03-19T23:11:21.378392+0000","last_peered":"2020-03-19T23:11:21.378392+0000","last_clean":"2020-03-19T23:11:21.378392+0000","last_became_active":"2020-03-19T23:11:04.283884+0000","last_became_peered":"2020-03-19T23:11:04.283884+0000","last_unstale":"2020-03-19T23:11:21.378392+0000","last_undegraded":"2020-03-19T23:11:21.378392+0000","last_fullsized":"2020-03-19T23:11:21.378392+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.646067+0000","last_change":"2020-03-19T23:11:04.652133+0000","last_active":"2020-03-19T23:11:20.646067+0000","last_peered":"2020-03-19T23:11:20.646067+0000","last_clean":"2020-03-19T23:11:20.646067+0000","last_became_active":"2020-03-19T23:11:04.651761+0000","last_became_peered":"2020-03-19T23:11:04.651761+0000","last_unstale":"2020-03-19T23:11:20.646067+0000","last_undegraded":"2020-03-19T23:11:20.646067+0000","last_fullsized":"2020-03-19T23:11:20.646067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370542+0000","last_change":"2020-03-19T23:11:01.250680+0000","last_active":"2020-03-19T23:11:20.370542+0000","last_peered":"2020-03-19T23:11:20.370542+0000","last_clean":"2020-03-19T23:11:20.370542+0000","last_became_active":"2020-03-19T23:11:01.250171+0000","last_became_peered":"2020-03-19T23:11:01.250171+0000","last_unstale":"2020-03-19T23:11:20.370542+0000","last_undegraded":"2020-03-19T23:11:20.370542+0000","last_fullsized":"2020-03-19T23:11:20.370542+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.646260+0000","last_change":"2020-03-19T23:11:04.287858+0000","last_active":"2020-03-19T23:11:20.646260+0000","last_peered":"2020-03-19T23:11:20.646260+0000","last_clean":"2020-03-19T23:11:20.646260+0000","last_became_active":"2020-03-19T23:11:04.287457+0000","last_became_peered":"2020-03-19T23:11:04.287457+0000","last_unstale":"2020-03-19T23:11:20.646260+0000","last_undegraded":"2020-03-19T23:11:20.646260+0000","last_fullsized":"2020-03-19T23:11:20.646260+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.646398+0000","last_change":"2020-03-19T23:11:16.353357+0000","last_active":"2020-03-19T23:11:20.646398+0000","last_peered":"2020-03-19T23:11:20.646398+0000","last_clean":"2020-03-19T23:11:20.646398+0000","last_became_active":"2020-03-19T23:11:16.352787+0000","last_became_peered":"2020-03-19T23:11:16.352787+0000","last_unstale":"2020-03-19T23:11:20.646398+0000","last_undegraded":"2020-03-19T23:11:20.646398+0000","last_fullsized":"2020-03-19T23:11:20.646398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378416+0000","last_change":"2020-03-19T23:11:04.654429+0000","last_active":"2020-03-19T23:11:21.378416+0000","last_peered":"2020-03-19T23:11:21.378416+0000","last_clean":"2020-03-19T23:11:21.378416+0000","last_became_active":"2020-03-19T23:11:04.654078+0000","last_became_peered":"2020-03-19T23:11:04.654078+0000","last_unstale":"2020-03-19T23:11:21.378416+0000","last_undegraded":"2020-03-19T23:11:21.378416+0000","last_fullsized":"2020-03-19T23:11:21.378416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378728+0000","last_change":"2020-03-19T23:11:01.252623+0000","last_active":"2020-03-19T23:11:21.378728+0000","last_peered":"2020-03-19T23:11:21.378728+0000","last_clean":"2020-03-19T23:11:21.378728+0000","last_became_active":"2020-03-19T23:11:01.252205+0000","last_became_peered":"2020-03-19T23:11:01.252205+0000","last_unstale":"2020-03-19T23:11:21.378728+0000","last_undegraded":"2020-03-19T23:11:21.378728+0000","last_fullsized":"2020-03-19T23:11:21.378728+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.379017+0000","last_change":"2020-03-19T23:11:04.285188+0000","last_active":"2020-03-19T23:11:21.379017+0000","last_peered":"2020-03-19T23:11:21.379017+0000","last_clean":"2020-03-19T23:11:21.379017+0000","last_became_active":"2020-03-19T23:11:04.284579+0000","last_became_peered":"2020-03-19T23:11:04.284579+0000","last_unstale":"2020-03-19T23:11:21.379017+0000","last_undegraded":"2020-03-19T23:11:21.379017+0000","last_fullsized":"2020-03-19T23:11:21.379017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378593+0000","last_change":"2020-03-19T23:11:16.354554+0000","last_active":"2020-03-19T23:11:21.378593+0000","last_peered":"2020-03-19T23:11:21.378593+0000","last_clean":"2020-03-19T23:11:21.378593+0000","last_became_active":"2020-03-19T23:11:16.354173+0000","last_became_peered":"2020-03-19T23:11:16.354173+0000","last_unstale":"2020-03-19T23:11:21.378593+0000","last_undegraded":"2020-03-19T23:11:21.378593+0000","last_fullsized":"2020-03-19T23:11:21.378593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370717+0000","last_change":"2020-03-19T23:11:04.650468+0000","last_active":"2020-03-19T23:11:20.370717+0000","last_peered":"2020-03-19T23:11:20.370717+0000","last_clean":"2020-03-19T23:11:20.370717+0000","last_became_active":"2020-03-19T23:11:04.650124+0000","last_became_peered":"2020-03-19T23:11:04.650124+0000","last_unstale":"2020-03-19T23:11:20.370717+0000","last_undegraded":"2020-03-19T23:11:20.370717+0000","last_fullsized":"2020-03-19T23:11:20.370717+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"19","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378684+0000","last_change":"2020-03-19T23:11:01.249394+0000","last_active":"2020-03-19T23:11:21.378684+0000","last_peered":"2020-03-19T23:11:21.378684+0000","last_clean":"2020-03-19T23:11:21.378684+0000","last_became_active":"2020-03-19T23:11:01.249024+0000","last_became_peered":"2020-03-19T23:11:01.249024+0000","last_unstale":"2020-03-19T23:11:21.378684+0000","last_undegraded":"2020-03-19T23:11:21.378684+0000","last_fullsized":"2020-03-19T23:11:21.378684+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:00.238767+0000","last_clean_scrub_stamp":"2020-03-19T23:11:00.238767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'10","reported_seq":"27","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:24.705355+0000","last_change":"2020-03-19T23:11:04.283517+0000","last_active":"2020-03-19T23:11:24.705355+0000","last_peered":"2020-03-19T23:11:24.705355+0000","last_clean":"2020-03-19T23:11:24.705355+0000","last_became_active":"2020-03-19T23:11:04.283065+0000","last_became_peered":"2020-03-19T23:11:04.283065+0000","last_unstale":"2020-03-19T23:11:24.705355+0000","last_undegraded":"2020-03-19T23:11:24.705355+0000","last_fullsized":"2020-03-19T23:11:24.705355+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1958,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738377,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808812,"statfs":{"total":107374182400,"available":106300223488,"internally_reserved":1073741824,"allocated":208896,"data_stored":77785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808788,"statfs":{"total":107374182400,"available":106300198912,"internally_reserved":1073741824,"allocated":233472,"data_stored":80435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808784,"statfs":{"total":107374182400,"available":106300194816,"internally_reserved":1073741824,"allocated":237568,"data_stored":80469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]}]}} 2020-03-19T23:11:29.527 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2020-03-19T23:11:15.327801+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=15, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.527 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.527 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.527 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.527 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2020-03-19T23:11:15.327801+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=15, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.527 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.528 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.528 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.528 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.528 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.528 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.529 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.529 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.529 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.529 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.529 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.529 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.530 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.530 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.530 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.530 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2020-03-19T23:11:15.327801+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=15, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.530 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.530 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.531 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.531 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2020-03-19T23:11:15.327801+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=15, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.531 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.531 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2020-03-19T23:11:00.238767+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=0, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.531 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:29.531 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-03-19T23:11:49.532 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T23:11:49.661 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.658+0000 7f1db9d9e700 1 Processor -- start 2020-03-19T23:11:49.661 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.658+0000 7f1db9d9e700 1 -- start start 2020-03-19T23:11:49.661 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db9d9e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db408e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:11:49.662 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db9d9e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db4060a40 con 0x7f1db40903e0 2020-03-19T23:11:49.662 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db9d9e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db4060500 con 0x7f1db4090ed0 2020-03-19T23:11:49.662 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db408e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:49.662 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db8d9c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1db40903e0 0x7f1db40907c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44378/0 (socket says 172.21.15.38:44378) 2020-03-19T23:11:49.662 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db8d9c700 1 -- 172.21.15.38:0/3358826351 learned_addr learned my addr 172.21.15.38:0/3358826351 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:49.662 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 -- 172.21.15.38:0/3358826351 >> v1:172.21.15.38:6789/0 conn(0x7f1db40903e0 legacy=0x7f1db40907c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:49.663 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db37fe700 1 -- 172.21.15.38:0/3358826351 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943393533 0 0) 0x7f1db4060a40 con 0x7f1db40903e0 2020-03-19T23:11:49.663 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 -- 172.21.15.38:0/3358826351 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4060730 con 0x7f1db4090ed0 2020-03-19T23:11:49.663 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 --2- 172.21.15.38:0/3358826351 >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db408e720 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.663 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 --2- 172.21.15.38:0/3358826351 >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db408e720 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.663 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 --2- 172.21.15.38:0/3358826351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db408e720 secure :-1 s=READY pgs=299 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).ready entity=mon.0 client_cookie=ce4775df31fc938a server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:49.663 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 --2- 172.21.15.38:0/3358826351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db408e720 secure :-1 s=THROTTLE_DONE pgs=299 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.663 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db37fe700 1 -- 172.21.15.38:0/3358826351 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1da0008e80 con 0x7f1db4090ed0 2020-03-19T23:11:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 --2- 172.21.15.38:0/3358826351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db408e720 secure :-1 s=THROTTLE_DONE pgs=299 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db37fe700 1 -- 172.21.15.38:0/3358826351 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da00113f0 con 0x7f1db4090ed0 2020-03-19T23:11:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db3fff700 1 --2- 172.21.15.38:0/3358826351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db408e720 secure :-1 s=THROTTLE_DONE pgs=299 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db9d9e700 1 -- 172.21.15.38:0/3358826351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 msgr2=0x7f1db408e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db37fe700 1 -- 172.21.15.38:0/3358826351 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1da00116d0 con 0x7f1db4090ed0 2020-03-19T23:11:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.659+0000 7f1db9d9e700 1 --2- 172.21.15.38:0/3358826351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db408e720 secure :-1 s=READY pgs=299 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).stop 2020-03-19T23:11:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.660+0000 7f1db9d9e700 1 -- 172.21.15.38:0/3358826351 shutdown_connections 2020-03-19T23:11:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.660+0000 7f1db9d9e700 1 --2- 172.21.15.38:0/3358826351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db408e720 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.660+0000 7f1db9d9e700 1 -- 172.21.15.38:0/3358826351 >> 172.21.15.38:0/3358826351 conn(0x7f1db40953a0 msgr2=0x7f1db40914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.660+0000 7f1db9d9e700 1 -- 172.21.15.38:0/3358826351 shutdown_connections 2020-03-19T23:11:49.665 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.660+0000 7f1db9d9e700 1 -- 172.21.15.38:0/3358826351 wait complete. 2020-03-19T23:11:49.668 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db9d9e700 1 Processor -- start 2020-03-19T23:11:49.668 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db9d9e700 1 -- start start 2020-03-19T23:11:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db9d9e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db4081190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db9d9e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db4088160 con 0x7f1db40903e0 2020-03-19T23:11:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db9d9e700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db4087c50 con 0x7f1db4090ed0 2020-03-19T23:11:49.669 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db4081190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:49.670 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db8d9c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1db40903e0 0x7f1db4080a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44382/0 (socket says 172.21.15.38:44382) 2020-03-19T23:11:49.670 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.661+0000 7f1db8d9c700 1 -- 172.21.15.38:0/4032344654 learned_addr learned my addr 172.21.15.38:0/4032344654 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:11:49.670 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 -- 172.21.15.38:0/4032344654 >> v1:172.21.15.38:6789/0 conn(0x7f1db40903e0 legacy=0x7f1db4080a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:49.670 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044947919 0 0) 0x7f1db4088160 con 0x7f1db40903e0 2020-03-19T23:11:49.670 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 -- 172.21.15.38:0/4032344654 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4081ba0 con 0x7f1db4090ed0 2020-03-19T23:11:49.670 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.670 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> v2:172.21.15.38:3300/0 conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=READY pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1da0011dc0 con 0x7f1db4090ed0 2020-03-19T23:11:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0007150 con 0x7f1db4090ed0 2020-03-19T23:11:49.671 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db4087e50 con 0x7f1db4090ed0 2020-03-19T23:11:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f1da00073f0 con 0x7f1db4090ed0 2020-03-19T23:11:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db40822e0 con 0x7f1db4090ed0 2020-03-19T23:11:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.662+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f1da00056c0 con 0x7f1db4090ed0 2020-03-19T23:11:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db1ffb700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:11:49.672 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f1da0034880 con 0x7f1db4090ed0 2020-03-19T23:11:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db8d9c700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:11:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db4056db0 con 0x7f1db4090ed0 2020-03-19T23:11:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db8d9c700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f1da8001260 tx=0x7f1da8001720).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db8d9c700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f1da8001260 tx=0x7f1da8001720).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.663+0000 7f1db8d9c700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7f1da8001260 tx=0x7f1da8001720).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:11:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.667+0000 7f1db3fff700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=THROTTLE_DONE pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.673 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.667+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f1da0006280 con 0x7f1db4090ed0 2020-03-19T23:11:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.810+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1db4081970 con 0x7f1d9c028250 2020-03-19T23:11:49.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.812+0000 7f1db8d9c700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 secure :-1 s=THROTTLE_DONE pgs=75 cs=0 l=1 rx=0x7f1da8001260 tx=0x7f1da8001720).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:11:49.814 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.812+0000 7f1db1ffb700 1 -- 172.21.15.38:0/4032344654 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+74084 (secure 0 0 0) 0x7f1db4081970 con 0x7f1d9c028250 2020-03-19T23:11:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:11:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 msgr2=0x7f1d9c02a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 secure :-1 s=READY pgs=75 cs=0 l=1 rx=0x7f1da8001260 tx=0x7f1da8001720).stop 2020-03-19T23:11:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 msgr2=0x7f1db4081190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:11:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 secure :-1 s=READY pgs=301 cs=0 l=1 rx=0x7f1da0002f30 tx=0x7f1da0007a40).stop 2020-03-19T23:11:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 shutdown_connections 2020-03-19T23:11:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f1d9c028250 0x7f1d9c02a710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:49.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 --2- 172.21.15.38:0/4032344654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1db4090ed0 0x7f1db4081190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:11:49.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 >> 172.21.15.38:0/4032344654 conn(0x7f1db40953a0 msgr2=0x7f1db4062bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:11:49.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 shutdown_connections 2020-03-19T23:11:49.817 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:11:49.814+0000 7f1db9d9e700 1 -- 172.21.15.38:0/4032344654 wait complete. 2020-03-19T23:11:49.817 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2020-03-19T23:11:49.825 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2020-03-19T23:11:49.623758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146416,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426384,"statfs":{"total":322122547200,"available":318900617216,"internally_reserved":3221225472,"allocated":679936,"data_stored":238689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002877"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:46.352178+0000","last_change":"2020-03-19T23:11:46.352249+0000","last_active":"2020-03-19T23:11:46.352178+0000","last_peered":"2020-03-19T23:11:46.352178+0000","last_clean":"2020-03-19T23:11:46.352178+0000","last_became_active":"2020-03-19T23:11:16.353972+0000","last_became_peered":"2020-03-19T23:11:16.353972+0000","last_unstale":"2020-03-19T23:11:46.352178+0000","last_undegraded":"2020-03-19T23:11:46.352178+0000","last_fullsized":"2020-03-19T23:11:46.352178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:46.352172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:46.352172+0000","last_clean_scrub_stamp":"2020-03-19T23:11:46.352172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370689+0000","last_change":"2020-03-19T23:11:04.653879+0000","last_active":"2020-03-19T23:11:20.370689+0000","last_peered":"2020-03-19T23:11:20.370689+0000","last_clean":"2020-03-19T23:11:20.370689+0000","last_became_active":"2020-03-19T23:11:04.653521+0000","last_became_peered":"2020-03-19T23:11:04.653521+0000","last_unstale":"2020-03-19T23:11:20.370689+0000","last_undegraded":"2020-03-19T23:11:20.370689+0000","last_fullsized":"2020-03-19T23:11:20.370689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:37.675742+0000","last_change":"2020-03-19T23:11:37.675815+0000","last_active":"2020-03-19T23:11:37.675742+0000","last_peered":"2020-03-19T23:11:37.675742+0000","last_clean":"2020-03-19T23:11:37.675742+0000","last_became_active":"2020-03-19T23:11:01.250084+0000","last_became_peered":"2020-03-19T23:11:01.250084+0000","last_unstale":"2020-03-19T23:11:37.675742+0000","last_undegraded":"2020-03-19T23:11:37.675742+0000","last_fullsized":"2020-03-19T23:11:37.675742+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:37.675732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:37.675732+0000","last_clean_scrub_stamp":"2020-03-19T23:11:37.675732+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"16'4","reported_seq":"21","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378847+0000","last_change":"2020-03-19T23:11:04.285615+0000","last_active":"2020-03-19T23:11:21.378847+0000","last_peered":"2020-03-19T23:11:21.378847+0000","last_clean":"2020-03-19T23:11:21.378847+0000","last_became_active":"2020-03-19T23:11:04.284829+0000","last_became_peered":"2020-03-19T23:11:04.284829+0000","last_unstale":"2020-03-19T23:11:21.378847+0000","last_undegraded":"2020-03-19T23:11:21.378847+0000","last_fullsized":"2020-03-19T23:11:21.378847+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.370486+0000","last_change":"2020-03-19T23:11:16.355229+0000","last_active":"2020-03-19T23:11:20.370486+0000","last_peered":"2020-03-19T23:11:20.370486+0000","last_clean":"2020-03-19T23:11:20.370486+0000","last_became_active":"2020-03-19T23:11:16.354846+0000","last_became_peered":"2020-03-19T23:11:16.354846+0000","last_unstale":"2020-03-19T23:11:20.370486+0000","last_undegraded":"2020-03-19T23:11:20.370486+0000","last_fullsized":"2020-03-19T23:11:20.370486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:41.389916+0000","last_change":"2020-03-19T23:11:41.389983+0000","last_active":"2020-03-19T23:11:41.389916+0000","last_peered":"2020-03-19T23:11:41.389916+0000","last_clean":"2020-03-19T23:11:41.389916+0000","last_became_active":"2020-03-19T23:11:04.653741+0000","last_became_peered":"2020-03-19T23:11:04.653741+0000","last_unstale":"2020-03-19T23:11:41.389916+0000","last_undegraded":"2020-03-19T23:11:41.389916+0000","last_fullsized":"2020-03-19T23:11:41.389916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:41.389910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:41.389910+0000","last_clean_scrub_stamp":"2020-03-19T23:11:41.389910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:36.909686+0000","last_change":"2020-03-19T23:11:36.909738+0000","last_active":"2020-03-19T23:11:36.909686+0000","last_peered":"2020-03-19T23:11:36.909686+0000","last_clean":"2020-03-19T23:11:36.909686+0000","last_became_active":"2020-03-19T23:11:01.251378+0000","last_became_peered":"2020-03-19T23:11:01.251378+0000","last_unstale":"2020-03-19T23:11:36.909686+0000","last_undegraded":"2020-03-19T23:11:36.909686+0000","last_fullsized":"2020-03-19T23:11:36.909686+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:36.909680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:36.909680+0000","last_clean_scrub_stamp":"2020-03-19T23:11:36.909680+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"16'7","reported_seq":"24","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378816+0000","last_change":"2020-03-19T23:11:04.285389+0000","last_active":"2020-03-19T23:11:21.378816+0000","last_peered":"2020-03-19T23:11:21.378816+0000","last_clean":"2020-03-19T23:11:21.378816+0000","last_became_active":"2020-03-19T23:11:04.284979+0000","last_became_peered":"2020-03-19T23:11:04.284979+0000","last_unstale":"2020-03-19T23:11:21.378816+0000","last_undegraded":"2020-03-19T23:11:21.378816+0000","last_fullsized":"2020-03-19T23:11:21.378816+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:37.366901+0000","last_change":"2020-03-19T23:11:37.366969+0000","last_active":"2020-03-19T23:11:37.366901+0000","last_peered":"2020-03-19T23:11:37.366901+0000","last_clean":"2020-03-19T23:11:37.366901+0000","last_became_active":"2020-03-19T23:11:04.649316+0000","last_became_peered":"2020-03-19T23:11:04.649316+0000","last_unstale":"2020-03-19T23:11:37.366901+0000","last_undegraded":"2020-03-19T23:11:37.366901+0000","last_fullsized":"2020-03-19T23:11:37.366901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:37.366891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:37.366891+0000","last_clean_scrub_stamp":"2020-03-19T23:11:37.366891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:35.955821+0000","last_change":"2020-03-19T23:11:35.955882+0000","last_active":"2020-03-19T23:11:35.955821+0000","last_peered":"2020-03-19T23:11:35.955821+0000","last_clean":"2020-03-19T23:11:35.955821+0000","last_became_active":"2020-03-19T23:11:01.248513+0000","last_became_peered":"2020-03-19T23:11:01.248513+0000","last_unstale":"2020-03-19T23:11:35.955821+0000","last_undegraded":"2020-03-19T23:11:35.955821+0000","last_fullsized":"2020-03-19T23:11:35.955821+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:35.955816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:35.955816+0000","last_clean_scrub_stamp":"2020-03-19T23:11:35.955816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"24'12","reported_seq":"31","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:38.703793+0000","last_change":"2020-03-19T23:11:38.703845+0000","last_active":"2020-03-19T23:11:38.703793+0000","last_peered":"2020-03-19T23:11:38.703793+0000","last_clean":"2020-03-19T23:11:38.703793+0000","last_became_active":"2020-03-19T23:11:04.287139+0000","last_became_peered":"2020-03-19T23:11:04.287139+0000","last_unstale":"2020-03-19T23:11:38.703793+0000","last_undegraded":"2020-03-19T23:11:38.703793+0000","last_fullsized":"2020-03-19T23:11:38.703793+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'12","last_scrub_stamp":"2020-03-19T23:11:38.703787+0000","last_deep_scrub":"24'12","last_deep_scrub_stamp":"2020-03-19T23:11:38.703787+0000","last_clean_scrub_stamp":"2020-03-19T23:11:38.703787+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378498+0000","last_change":"2020-03-19T23:11:04.654111+0000","last_active":"2020-03-19T23:11:21.378498+0000","last_peered":"2020-03-19T23:11:21.378498+0000","last_clean":"2020-03-19T23:11:21.378498+0000","last_became_active":"2020-03-19T23:11:04.653768+0000","last_became_peered":"2020-03-19T23:11:04.653768+0000","last_unstale":"2020-03-19T23:11:21.378498+0000","last_undegraded":"2020-03-19T23:11:21.378498+0000","last_fullsized":"2020-03-19T23:11:21.378498+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:29.304156+0000","last_change":"2020-03-19T23:11:29.304200+0000","last_active":"2020-03-19T23:11:29.304156+0000","last_peered":"2020-03-19T23:11:29.304156+0000","last_clean":"2020-03-19T23:11:29.304156+0000","last_became_active":"2020-03-19T23:11:01.251474+0000","last_became_peered":"2020-03-19T23:11:01.251474+0000","last_unstale":"2020-03-19T23:11:29.304156+0000","last_undegraded":"2020-03-19T23:11:29.304156+0000","last_fullsized":"2020-03-19T23:11:29.304156+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:29.304152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:29.304152+0000","last_clean_scrub_stamp":"2020-03-19T23:11:29.304152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:40.700908+0000","last_change":"2020-03-19T23:11:40.700962+0000","last_active":"2020-03-19T23:11:40.700908+0000","last_peered":"2020-03-19T23:11:40.700908+0000","last_clean":"2020-03-19T23:11:40.700908+0000","last_became_active":"2020-03-19T23:11:04.286797+0000","last_became_peered":"2020-03-19T23:11:04.286797+0000","last_unstale":"2020-03-19T23:11:40.700908+0000","last_undegraded":"2020-03-19T23:11:40.700908+0000","last_fullsized":"2020-03-19T23:11:40.700908+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:40.700904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:40.700904+0000","last_clean_scrub_stamp":"2020-03-19T23:11:40.700904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378901+0000","last_change":"2020-03-19T23:11:04.650882+0000","last_active":"2020-03-19T23:11:21.378901+0000","last_peered":"2020-03-19T23:11:21.378901+0000","last_clean":"2020-03-19T23:11:21.378901+0000","last_became_active":"2020-03-19T23:11:04.650546+0000","last_became_peered":"2020-03-19T23:11:04.650546+0000","last_unstale":"2020-03-19T23:11:21.378901+0000","last_undegraded":"2020-03-19T23:11:21.378901+0000","last_fullsized":"2020-03-19T23:11:21.378901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:34.031351+0000","last_change":"2020-03-19T23:11:34.031428+0000","last_active":"2020-03-19T23:11:34.031351+0000","last_peered":"2020-03-19T23:11:34.031351+0000","last_clean":"2020-03-19T23:11:34.031351+0000","last_became_active":"2020-03-19T23:11:01.249356+0000","last_became_peered":"2020-03-19T23:11:01.249356+0000","last_unstale":"2020-03-19T23:11:34.031351+0000","last_undegraded":"2020-03-19T23:11:34.031351+0000","last_fullsized":"2020-03-19T23:11:34.031351+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:34.031344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:34.031344+0000","last_clean_scrub_stamp":"2020-03-19T23:11:34.031344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"24'8","reported_seq":"28","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:42.869414+0000","last_change":"2020-03-19T23:11:42.869468+0000","last_active":"2020-03-19T23:11:42.869414+0000","last_peered":"2020-03-19T23:11:42.869414+0000","last_clean":"2020-03-19T23:11:42.869414+0000","last_became_active":"2020-03-19T23:11:04.283884+0000","last_became_peered":"2020-03-19T23:11:04.283884+0000","last_unstale":"2020-03-19T23:11:42.869414+0000","last_undegraded":"2020-03-19T23:11:42.869414+0000","last_fullsized":"2020-03-19T23:11:42.869414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'8","last_scrub_stamp":"2020-03-19T23:11:42.869407+0000","last_deep_scrub":"24'8","last_deep_scrub_stamp":"2020-03-19T23:11:42.869407+0000","last_clean_scrub_stamp":"2020-03-19T23:11:42.869407+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:39.387377+0000","last_change":"2020-03-19T23:11:39.387432+0000","last_active":"2020-03-19T23:11:39.387377+0000","last_peered":"2020-03-19T23:11:39.387377+0000","last_clean":"2020-03-19T23:11:39.387377+0000","last_became_active":"2020-03-19T23:11:04.651761+0000","last_became_peered":"2020-03-19T23:11:04.651761+0000","last_unstale":"2020-03-19T23:11:39.387377+0000","last_undegraded":"2020-03-19T23:11:39.387377+0000","last_fullsized":"2020-03-19T23:11:39.387377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:39.387371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:39.387371+0000","last_clean_scrub_stamp":"2020-03-19T23:11:39.387371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:30.732884+0000","last_change":"2020-03-19T23:11:30.732947+0000","last_active":"2020-03-19T23:11:30.732884+0000","last_peered":"2020-03-19T23:11:30.732884+0000","last_clean":"2020-03-19T23:11:30.732884+0000","last_became_active":"2020-03-19T23:11:01.250171+0000","last_became_peered":"2020-03-19T23:11:01.250171+0000","last_unstale":"2020-03-19T23:11:30.732884+0000","last_undegraded":"2020-03-19T23:11:30.732884+0000","last_fullsized":"2020-03-19T23:11:30.732884+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:30.732878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:30.732878+0000","last_clean_scrub_stamp":"2020-03-19T23:11:30.732878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:36.410635+0000","last_change":"2020-03-19T23:11:36.410689+0000","last_active":"2020-03-19T23:11:36.410635+0000","last_peered":"2020-03-19T23:11:36.410635+0000","last_clean":"2020-03-19T23:11:36.410635+0000","last_became_active":"2020-03-19T23:11:04.287457+0000","last_became_peered":"2020-03-19T23:11:04.287457+0000","last_unstale":"2020-03-19T23:11:36.410635+0000","last_undegraded":"2020-03-19T23:11:36.410635+0000","last_fullsized":"2020-03-19T23:11:36.410635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2020-03-19T23:11:36.410629+0000","last_deep_scrub":"16'1","last_deep_scrub_stamp":"2020-03-19T23:11:36.410629+0000","last_clean_scrub_stamp":"2020-03-19T23:11:36.410629+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2020-03-19T23:11:20.646398+0000","last_change":"2020-03-19T23:11:16.353357+0000","last_active":"2020-03-19T23:11:20.646398+0000","last_peered":"2020-03-19T23:11:20.646398+0000","last_clean":"2020-03-19T23:11:20.646398+0000","last_became_active":"2020-03-19T23:11:16.352787+0000","last_became_peered":"2020-03-19T23:11:16.352787+0000","last_unstale":"2020-03-19T23:11:20.646398+0000","last_undegraded":"2020-03-19T23:11:20.646398+0000","last_fullsized":"2020-03-19T23:11:20.646398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"16","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378416+0000","last_change":"2020-03-19T23:11:04.654429+0000","last_active":"2020-03-19T23:11:21.378416+0000","last_peered":"2020-03-19T23:11:21.378416+0000","last_clean":"2020-03-19T23:11:21.378416+0000","last_became_active":"2020-03-19T23:11:04.654078+0000","last_became_peered":"2020-03-19T23:11:04.654078+0000","last_unstale":"2020-03-19T23:11:21.378416+0000","last_undegraded":"2020-03-19T23:11:21.378416+0000","last_fullsized":"2020-03-19T23:11:21.378416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:04.269091+0000","last_clean_scrub_stamp":"2020-03-19T23:11:04.269091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:38.874554+0000","last_change":"2020-03-19T23:11:38.874610+0000","last_active":"2020-03-19T23:11:38.874554+0000","last_peered":"2020-03-19T23:11:38.874554+0000","last_clean":"2020-03-19T23:11:38.874554+0000","last_became_active":"2020-03-19T23:11:01.252205+0000","last_became_peered":"2020-03-19T23:11:01.252205+0000","last_unstale":"2020-03-19T23:11:38.874554+0000","last_undegraded":"2020-03-19T23:11:38.874554+0000","last_fullsized":"2020-03-19T23:11:38.874554+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:38.874549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:38.874549+0000","last_clean_scrub_stamp":"2020-03-19T23:11:38.874549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"16'6","reported_seq":"23","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.379017+0000","last_change":"2020-03-19T23:11:04.285188+0000","last_active":"2020-03-19T23:11:21.379017+0000","last_peered":"2020-03-19T23:11:21.379017+0000","last_clean":"2020-03-19T23:11:21.379017+0000","last_became_active":"2020-03-19T23:11:04.284579+0000","last_became_peered":"2020-03-19T23:11:04.284579+0000","last_unstale":"2020-03-19T23:11:21.379017+0000","last_undegraded":"2020-03-19T23:11:21.379017+0000","last_fullsized":"2020-03-19T23:11:21.379017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:03.263194+0000","last_clean_scrub_stamp":"2020-03-19T23:11:03.263194+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"13","reported_epoch":"23","state":"active+clean","last_fresh":"2020-03-19T23:11:21.378593+0000","last_change":"2020-03-19T23:11:16.354554+0000","last_active":"2020-03-19T23:11:21.378593+0000","last_peered":"2020-03-19T23:11:21.378593+0000","last_clean":"2020-03-19T23:11:21.378593+0000","last_became_active":"2020-03-19T23:11:16.354173+0000","last_became_peered":"2020-03-19T23:11:16.354173+0000","last_unstale":"2020-03-19T23:11:21.378593+0000","last_undegraded":"2020-03-19T23:11:21.378593+0000","last_fullsized":"2020-03-19T23:11:21.378593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:15.327801+0000","last_clean_scrub_stamp":"2020-03-19T23:11:15.327801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:42.745317+0000","last_change":"2020-03-19T23:11:42.745375+0000","last_active":"2020-03-19T23:11:42.745317+0000","last_peered":"2020-03-19T23:11:42.745317+0000","last_clean":"2020-03-19T23:11:42.745317+0000","last_became_active":"2020-03-19T23:11:04.650124+0000","last_became_peered":"2020-03-19T23:11:04.650124+0000","last_unstale":"2020-03-19T23:11:42.745317+0000","last_undegraded":"2020-03-19T23:11:42.745317+0000","last_fullsized":"2020-03-19T23:11:42.745317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:42.745302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:42.745302+0000","last_clean_scrub_stamp":"2020-03-19T23:11:42.745302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:41.826931+0000","last_change":"2020-03-19T23:11:41.826984+0000","last_active":"2020-03-19T23:11:41.826931+0000","last_peered":"2020-03-19T23:11:41.826931+0000","last_clean":"2020-03-19T23:11:41.826931+0000","last_became_active":"2020-03-19T23:11:01.249024+0000","last_became_peered":"2020-03-19T23:11:01.249024+0000","last_unstale":"2020-03-19T23:11:41.826931+0000","last_undegraded":"2020-03-19T23:11:41.826931+0000","last_fullsized":"2020-03-19T23:11:41.826931+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:41.826925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:41.826925+0000","last_clean_scrub_stamp":"2020-03-19T23:11:41.826925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'13","reported_seq":"33","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:46.907519+0000","last_change":"2020-03-19T23:11:46.907579+0000","last_active":"2020-03-19T23:11:46.907519+0000","last_peered":"2020-03-19T23:11:46.907519+0000","last_clean":"2020-03-19T23:11:46.907519+0000","last_became_active":"2020-03-19T23:11:04.283065+0000","last_became_peered":"2020-03-19T23:11:04.283065+0000","last_unstale":"2020-03-19T23:11:46.907519+0000","last_undegraded":"2020-03-19T23:11:46.907519+0000","last_fullsized":"2020-03-19T23:11:46.907519+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'13","last_scrub_stamp":"2020-03-19T23:11:46.907513+0000","last_deep_scrub":"24'13","last_deep_scrub_stamp":"2020-03-19T23:11:46.907513+0000","last_clean_scrub_stamp":"2020-03-19T23:11:46.907513+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1958,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738381,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808812,"statfs":{"total":107374182400,"available":106300223488,"internally_reserved":1073741824,"allocated":208896,"data_stored":77785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738380,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808788,"statfs":{"total":107374182400,"available":106300198912,"internally_reserved":1073741824,"allocated":233472,"data_stored":80435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738380,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808784,"statfs":{"total":107374182400,"available":106300194816,"internally_reserved":1073741824,"allocated":237568,"data_stored":80469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}]}} 2020-03-19T23:11:49.828 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.828 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.829 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2020-03-19T23:11:15.327801+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=15, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.829 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.829 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.829 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.829 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2020-03-19T23:11:15.327801+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=15, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.830 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2020-03-19T23:11:04.269091+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=4, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.830 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-03-19T23:11:03.263194+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=3, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.830 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2020-03-19T23:11:15.327801+0000 time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=15, tm_wday=3, tm_yday=79, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=3, tm_mday=19, tm_hour=23, tm_min=11, tm_sec=26, tm_wday=3, tm_yday=79, tm_isdst=0) 2020-03-19T23:11:49.830 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2020-03-19T23:12:09.831 INFO:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-03-19T23:12:09.958 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.956+0000 7f81f48f0700 1 Processor -- start 2020-03-19T23:12:09.959 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.957+0000 7f81f48f0700 1 -- start start 2020-03-19T23:12:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.957+0000 7f81f48f0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f81f01221d0 0x7f81f01225b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:12:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.957+0000 7f81f48f0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f008eee0 con 0x7f81f0122a10 2020-03-19T23:12:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.957+0000 7f81f48f0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f008dda0 con 0x7f81f01221d0 2020-03-19T23:12:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.957+0000 7f81ee59c700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f81f01221d0 0x7f81f01225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:12:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.957+0000 7f81edd9b700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f81f0122a10 0x7f81f008bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44394/0 (socket says 172.21.15.38:44394) 2020-03-19T23:12:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.957+0000 7f81edd9b700 1 -- 172.21.15.38:0/3953312540 learned_addr learned my addr 172.21.15.38:0/3953312540 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:12:09.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 -- 172.21.15.38:0/3953312540 >> v1:172.21.15.38:6789/0 conn(0x7f81f0122a10 legacy=0x7f81f008bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:09.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 -- 172.21.15.38:0/3953312540 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0121fd0 con 0x7f81f01221d0 2020-03-19T23:12:09.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ed59a700 1 -- 172.21.15.38:0/3953312540 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285830912 0 0) 0x7f81f008eee0 con 0x7f81f0122a10 2020-03-19T23:12:09.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 --2- 172.21.15.38:0/3953312540 >> v2:172.21.15.38:3300/0 conn(0x7f81f01221d0 0x7f81f01225b0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7f81d8002f30 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.968 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 --2- 172.21.15.38:0/3953312540 >> v2:172.21.15.38:3300/0 conn(0x7f81f01221d0 0x7f81f01225b0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7f81d8002f30 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 --2- 172.21.15.38:0/3953312540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f01221d0 0x7f81f01225b0 secure :-1 s=READY pgs=303 cs=0 l=1 rx=0x7f81d8002f30 tx=0x7f81d8007a40).ready entity=mon.0 client_cookie=d9b3580433e52e99 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:12:09.969 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 --2- 172.21.15.38:0/3953312540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f01221d0 0x7f81f01225b0 secure :-1 s=THROTTLE_DONE pgs=303 cs=0 l=1 rx=0x7f81d8002f30 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ed59a700 1 -- 172.21.15.38:0/3953312540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f81d80113f0 con 0x7f81f01221d0 2020-03-19T23:12:09.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 --2- 172.21.15.38:0/3953312540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f01221d0 0x7f81f01225b0 secure :-1 s=THROTTLE_DONE pgs=303 cs=0 l=1 rx=0x7f81d8002f30 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.970 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ee59c700 1 --2- 172.21.15.38:0/3953312540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f01221d0 0x7f81f01225b0 secure :-1 s=THROTTLE_DONE pgs=303 cs=0 l=1 rx=0x7f81d8002f30 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ed59a700 1 -- 172.21.15.38:0/3953312540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d8011550 con 0x7f81f01221d0 2020-03-19T23:12:09.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81ed59a700 1 -- 172.21.15.38:0/3953312540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f81d80117f0 con 0x7f81f01221d0 2020-03-19T23:12:09.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81f48f0700 1 -- 172.21.15.38:0/3953312540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f01221d0 msgr2=0x7f81f01225b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:09.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81f48f0700 1 --2- 172.21.15.38:0/3953312540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f01221d0 0x7f81f01225b0 secure :-1 s=READY pgs=303 cs=0 l=1 rx=0x7f81d8002f30 tx=0x7f81d8007a40).stop 2020-03-19T23:12:09.971 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81f48f0700 1 -- 172.21.15.38:0/3953312540 shutdown_connections 2020-03-19T23:12:09.972 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81f48f0700 1 --2- 172.21.15.38:0/3953312540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f01221d0 0x7f81f01225b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:12:09.972 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81f48f0700 1 -- 172.21.15.38:0/3953312540 >> 172.21.15.38:0/3953312540 conn(0x7f81f0091540 msgr2=0x7f81f0091950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:12:09.972 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.958+0000 7f81f48f0700 1 -- 172.21.15.38:0/3953312540 shutdown_connections 2020-03-19T23:12:09.972 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.959+0000 7f81f48f0700 1 -- 172.21.15.38:0/3953312540 wait complete. 2020-03-19T23:12:09.973 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.959+0000 7f81f48f0700 1 Processor -- start 2020-03-19T23:12:09.973 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.959+0000 7f81f48f0700 1 -- start start 2020-03-19T23:12:09.973 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81f48f0700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f81f0122a10 0x7f81f0124690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:12:09.973 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81f48f0700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f00669c0 con 0x7f81f01221d0 2020-03-19T23:12:09.974 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81f48f0700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f00651d0 con 0x7f81f0122a10 2020-03-19T23:12:09.974 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f81f0122a10 0x7f81f0124690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:12:09.974 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81ee59c700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f81f01221d0 0x7f81f0065840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44396/0 (socket says 172.21.15.38:44396) 2020-03-19T23:12:09.974 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81ee59c700 1 -- 172.21.15.38:0/4218482555 learned_addr learned my addr 172.21.15.38:0/4218482555 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:12:09.975 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 -- 172.21.15.38:0/4218482555 >> v1:172.21.15.38:6789/0 conn(0x7f81f01221d0 legacy=0x7f81f0065840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:09.975 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974315930 0 0) 0x7f81f00669c0 con 0x7f81f01221d0 2020-03-19T23:12:09.976 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 -- 172.21.15.38:0/4218482555 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0124af0 con 0x7f81f0122a10 2020-03-19T23:12:09.976 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> v2:172.21.15.38:3300/0 conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.976 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> v2:172.21.15.38:3300/0 conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.976 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=READY pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:12:09.977 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.977 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f81e0005b20 con 0x7f81f0122a10 2020-03-19T23:12:09.977 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.977 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.978 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e0005c80 con 0x7f81f0122a10 2020-03-19T23:12:09.978 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7f81e00060c0 con 0x7f81f0122a10 2020-03-19T23:12:09.978 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.960+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f008d410 con 0x7f81f0122a10 2020-03-19T23:12:09.978 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.961+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f0123030 con 0x7f81f0122a10 2020-03-19T23:12:09.979 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.961+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.979 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.961+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7f81e000a320 con 0x7f81f0122a10 2020-03-19T23:12:09.979 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.961+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.980 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.961+0000 7f81e77fe700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:12:09.980 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.961+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7f81e0034020 con 0x7f81f0122a10 2020-03-19T23:12:09.980 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.961+0000 7f81ee59c700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:12:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.962+0000 7f81ee59c700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7f81d80101c0 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.962+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f0056db0 con 0x7f81f0122a10 2020-03-19T23:12:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.962+0000 7f81ee59c700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7f81d80101c0 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.981 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.962+0000 7f81ee59c700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f81d80101c0 tx=0x7f81d8007a40).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:12:09.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.965+0000 7f81edd9b700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=THROTTLE_DONE pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:09.982 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:09.965+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7f81e0006220 con 0x7f81f0122a10 2020-03-19T23:12:10.112 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.110+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 --> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81f00670e0 con 0x7f81d4028250 2020-03-19T23:12:10.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.112+0000 7f81ee59c700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 secure :-1 s=THROTTLE_DONE pgs=76 cs=0 l=1 rx=0x7f81d80101c0 tx=0x7f81d8007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.115 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.112+0000 7f81e77fe700 1 -- 172.21.15.38:0/4218482555 <== mgr.4103 v2:172.21.15.38:6824/20200 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+76605 (secure 0 0 0) 0x7f81d8008660 con 0x7f81d4028250 2020-03-19T23:12:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:10.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 msgr2=0x7f81d402a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:10.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 secure :-1 s=READY pgs=76 cs=0 l=1 rx=0x7f81d80101c0 tx=0x7f81d8007a40).stop 2020-03-19T23:12:10.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 msgr2=0x7f81f0124690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:10.117 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 secure :-1 s=READY pgs=305 cs=0 l=1 rx=0x7f81e0000b80 tx=0x7f81e0010950).stop 2020-03-19T23:12:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 shutdown_connections 2020-03-19T23:12:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7f81d4028250 0x7f81d402a710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:12:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 --2- 172.21.15.38:0/4218482555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81f0122a10 0x7f81f0124690 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:12:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 >> 172.21.15.38:0/4218482555 conn(0x7f81f0091540 msgr2=0x7f81f0093e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:12:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 shutdown_connections 2020-03-19T23:12:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.114+0000 7f81f48f0700 1 -- 172.21.15.38:0/4218482555 wait complete. 2020-03-19T23:12:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2020-03-19T23:12:10.127 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2020-03-19T23:12:09.628248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":3146416,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":3145728,"kb_avail":311426384,"statfs":{"total":322122547200,"available":318900617216,"internally_reserved":3221225472,"allocated":679936,"data_stored":238689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":3221225472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002920"},"pg_stats":[{"pgid":"4.0","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:46.352178+0000","last_change":"2020-03-19T23:11:46.352249+0000","last_active":"2020-03-19T23:11:46.352178+0000","last_peered":"2020-03-19T23:11:46.352178+0000","last_clean":"2020-03-19T23:11:46.352178+0000","last_became_active":"2020-03-19T23:11:16.353972+0000","last_became_peered":"2020-03-19T23:11:16.353972+0000","last_unstale":"2020-03-19T23:11:46.352178+0000","last_undegraded":"2020-03-19T23:11:46.352178+0000","last_fullsized":"2020-03-19T23:11:46.352178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:46.352172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:46.352172+0000","last_clean_scrub_stamp":"2020-03-19T23:11:46.352172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:47.714682+0000","last_change":"2020-03-19T23:11:47.714750+0000","last_active":"2020-03-19T23:11:47.714682+0000","last_peered":"2020-03-19T23:11:47.714682+0000","last_clean":"2020-03-19T23:11:47.714682+0000","last_became_active":"2020-03-19T23:11:04.653521+0000","last_became_peered":"2020-03-19T23:11:04.653521+0000","last_unstale":"2020-03-19T23:11:47.714682+0000","last_undegraded":"2020-03-19T23:11:47.714682+0000","last_fullsized":"2020-03-19T23:11:47.714682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:47.714678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:47.714678+0000","last_clean_scrub_stamp":"2020-03-19T23:11:47.714678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:37.675742+0000","last_change":"2020-03-19T23:11:37.675815+0000","last_active":"2020-03-19T23:11:37.675742+0000","last_peered":"2020-03-19T23:11:37.675742+0000","last_clean":"2020-03-19T23:11:37.675742+0000","last_became_active":"2020-03-19T23:11:01.250084+0000","last_became_peered":"2020-03-19T23:11:01.250084+0000","last_unstale":"2020-03-19T23:11:37.675742+0000","last_undegraded":"2020-03-19T23:11:37.675742+0000","last_fullsized":"2020-03-19T23:11:37.675742+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:37.675732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:37.675732+0000","last_clean_scrub_stamp":"2020-03-19T23:11:37.675732+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"24'6","reported_seq":"26","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:52.848781+0000","last_change":"2020-03-19T23:11:52.848837+0000","last_active":"2020-03-19T23:11:52.848781+0000","last_peered":"2020-03-19T23:11:52.848781+0000","last_clean":"2020-03-19T23:11:52.848781+0000","last_became_active":"2020-03-19T23:11:04.284829+0000","last_became_peered":"2020-03-19T23:11:04.284829+0000","last_unstale":"2020-03-19T23:11:52.848781+0000","last_undegraded":"2020-03-19T23:11:52.848781+0000","last_fullsized":"2020-03-19T23:11:52.848781+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6","last_scrub_stamp":"2020-03-19T23:11:52.848774+0000","last_deep_scrub":"24'6","last_deep_scrub_stamp":"2020-03-19T23:11:52.848774+0000","last_clean_scrub_stamp":"2020-03-19T23:11:52.848774+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:53.775287+0000","last_change":"2020-03-19T23:11:53.775366+0000","last_active":"2020-03-19T23:11:53.775287+0000","last_peered":"2020-03-19T23:11:53.775287+0000","last_clean":"2020-03-19T23:11:53.775287+0000","last_became_active":"2020-03-19T23:11:16.354846+0000","last_became_peered":"2020-03-19T23:11:16.354846+0000","last_unstale":"2020-03-19T23:11:53.775287+0000","last_undegraded":"2020-03-19T23:11:53.775287+0000","last_fullsized":"2020-03-19T23:11:53.775287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:53.775281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:53.775281+0000","last_clean_scrub_stamp":"2020-03-19T23:11:53.775281+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:41.389916+0000","last_change":"2020-03-19T23:11:41.389983+0000","last_active":"2020-03-19T23:11:41.389916+0000","last_peered":"2020-03-19T23:11:41.389916+0000","last_clean":"2020-03-19T23:11:41.389916+0000","last_became_active":"2020-03-19T23:11:04.653741+0000","last_became_peered":"2020-03-19T23:11:04.653741+0000","last_unstale":"2020-03-19T23:11:41.389916+0000","last_undegraded":"2020-03-19T23:11:41.389916+0000","last_fullsized":"2020-03-19T23:11:41.389916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:41.389910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:41.389910+0000","last_clean_scrub_stamp":"2020-03-19T23:11:41.389910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:36.909686+0000","last_change":"2020-03-19T23:11:36.909738+0000","last_active":"2020-03-19T23:11:36.909686+0000","last_peered":"2020-03-19T23:11:36.909686+0000","last_clean":"2020-03-19T23:11:36.909686+0000","last_became_active":"2020-03-19T23:11:01.251378+0000","last_became_peered":"2020-03-19T23:11:01.251378+0000","last_unstale":"2020-03-19T23:11:36.909686+0000","last_undegraded":"2020-03-19T23:11:36.909686+0000","last_fullsized":"2020-03-19T23:11:36.909686+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:36.909680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:36.909680+0000","last_clean_scrub_stamp":"2020-03-19T23:11:36.909680+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"24'11","reported_seq":"31","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:55.926208+0000","last_change":"2020-03-19T23:11:55.926259+0000","last_active":"2020-03-19T23:11:55.926208+0000","last_peered":"2020-03-19T23:11:55.926208+0000","last_clean":"2020-03-19T23:11:55.926208+0000","last_became_active":"2020-03-19T23:11:04.284979+0000","last_became_peered":"2020-03-19T23:11:04.284979+0000","last_unstale":"2020-03-19T23:11:55.926208+0000","last_undegraded":"2020-03-19T23:11:55.926208+0000","last_fullsized":"2020-03-19T23:11:55.926208+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'11","last_scrub_stamp":"2020-03-19T23:11:55.926201+0000","last_deep_scrub":"24'11","last_deep_scrub_stamp":"2020-03-19T23:11:55.926201+0000","last_clean_scrub_stamp":"2020-03-19T23:11:55.926201+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":566,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:37.366901+0000","last_change":"2020-03-19T23:11:37.366969+0000","last_active":"2020-03-19T23:11:37.366901+0000","last_peered":"2020-03-19T23:11:37.366901+0000","last_clean":"2020-03-19T23:11:37.366901+0000","last_became_active":"2020-03-19T23:11:04.649316+0000","last_became_peered":"2020-03-19T23:11:04.649316+0000","last_unstale":"2020-03-19T23:11:37.366901+0000","last_undegraded":"2020-03-19T23:11:37.366901+0000","last_fullsized":"2020-03-19T23:11:37.366901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:37.366891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:37.366891+0000","last_clean_scrub_stamp":"2020-03-19T23:11:37.366891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:35.955821+0000","last_change":"2020-03-19T23:11:35.955882+0000","last_active":"2020-03-19T23:11:35.955821+0000","last_peered":"2020-03-19T23:11:35.955821+0000","last_clean":"2020-03-19T23:11:35.955821+0000","last_became_active":"2020-03-19T23:11:01.248513+0000","last_became_peered":"2020-03-19T23:11:01.248513+0000","last_unstale":"2020-03-19T23:11:35.955821+0000","last_undegraded":"2020-03-19T23:11:35.955821+0000","last_fullsized":"2020-03-19T23:11:35.955821+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:35.955816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:35.955816+0000","last_clean_scrub_stamp":"2020-03-19T23:11:35.955816+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"24'12","reported_seq":"31","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:38.703793+0000","last_change":"2020-03-19T23:11:38.703845+0000","last_active":"2020-03-19T23:11:38.703793+0000","last_peered":"2020-03-19T23:11:38.703793+0000","last_clean":"2020-03-19T23:11:38.703793+0000","last_became_active":"2020-03-19T23:11:04.287139+0000","last_became_peered":"2020-03-19T23:11:04.287139+0000","last_unstale":"2020-03-19T23:11:38.703793+0000","last_undegraded":"2020-03-19T23:11:38.703793+0000","last_fullsized":"2020-03-19T23:11:38.703793+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'12","last_scrub_stamp":"2020-03-19T23:11:38.703787+0000","last_deep_scrub":"24'12","last_deep_scrub_stamp":"2020-03-19T23:11:38.703787+0000","last_clean_scrub_stamp":"2020-03-19T23:11:38.703787+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:57.944727+0000","last_change":"2020-03-19T23:11:57.944784+0000","last_active":"2020-03-19T23:11:57.944727+0000","last_peered":"2020-03-19T23:11:57.944727+0000","last_clean":"2020-03-19T23:11:57.944727+0000","last_became_active":"2020-03-19T23:11:04.653768+0000","last_became_peered":"2020-03-19T23:11:04.653768+0000","last_unstale":"2020-03-19T23:11:57.944727+0000","last_undegraded":"2020-03-19T23:11:57.944727+0000","last_fullsized":"2020-03-19T23:11:57.944727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:57.944720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:57.944720+0000","last_clean_scrub_stamp":"2020-03-19T23:11:57.944720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:29.304156+0000","last_change":"2020-03-19T23:11:29.304200+0000","last_active":"2020-03-19T23:11:29.304156+0000","last_peered":"2020-03-19T23:11:29.304156+0000","last_clean":"2020-03-19T23:11:29.304156+0000","last_became_active":"2020-03-19T23:11:01.251474+0000","last_became_peered":"2020-03-19T23:11:01.251474+0000","last_unstale":"2020-03-19T23:11:29.304156+0000","last_undegraded":"2020-03-19T23:11:29.304156+0000","last_fullsized":"2020-03-19T23:11:29.304156+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:29.304152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:29.304152+0000","last_clean_scrub_stamp":"2020-03-19T23:11:29.304152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:40.700908+0000","last_change":"2020-03-19T23:11:40.700962+0000","last_active":"2020-03-19T23:11:40.700908+0000","last_peered":"2020-03-19T23:11:40.700908+0000","last_clean":"2020-03-19T23:11:40.700908+0000","last_became_active":"2020-03-19T23:11:04.286797+0000","last_became_peered":"2020-03-19T23:11:04.286797+0000","last_unstale":"2020-03-19T23:11:40.700908+0000","last_undegraded":"2020-03-19T23:11:40.700908+0000","last_fullsized":"2020-03-19T23:11:40.700908+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:40.700904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:40.700904+0000","last_clean_scrub_stamp":"2020-03-19T23:11:40.700904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:59.969602+0000","last_change":"2020-03-19T23:11:59.969659+0000","last_active":"2020-03-19T23:11:59.969602+0000","last_peered":"2020-03-19T23:11:59.969602+0000","last_clean":"2020-03-19T23:11:59.969602+0000","last_became_active":"2020-03-19T23:11:04.650546+0000","last_became_peered":"2020-03-19T23:11:04.650546+0000","last_unstale":"2020-03-19T23:11:59.969602+0000","last_undegraded":"2020-03-19T23:11:59.969602+0000","last_fullsized":"2020-03-19T23:11:59.969602+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:59.969596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:59.969596+0000","last_clean_scrub_stamp":"2020-03-19T23:11:59.969596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:34.031351+0000","last_change":"2020-03-19T23:11:34.031428+0000","last_active":"2020-03-19T23:11:34.031351+0000","last_peered":"2020-03-19T23:11:34.031351+0000","last_clean":"2020-03-19T23:11:34.031351+0000","last_became_active":"2020-03-19T23:11:01.249356+0000","last_became_peered":"2020-03-19T23:11:01.249356+0000","last_unstale":"2020-03-19T23:11:34.031351+0000","last_undegraded":"2020-03-19T23:11:34.031351+0000","last_fullsized":"2020-03-19T23:11:34.031351+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:34.031344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:34.031344+0000","last_clean_scrub_stamp":"2020-03-19T23:11:34.031344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"24'8","reported_seq":"28","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:42.869414+0000","last_change":"2020-03-19T23:11:42.869468+0000","last_active":"2020-03-19T23:11:42.869414+0000","last_peered":"2020-03-19T23:11:42.869414+0000","last_clean":"2020-03-19T23:11:42.869414+0000","last_became_active":"2020-03-19T23:11:04.283884+0000","last_became_peered":"2020-03-19T23:11:04.283884+0000","last_unstale":"2020-03-19T23:11:42.869414+0000","last_undegraded":"2020-03-19T23:11:42.869414+0000","last_fullsized":"2020-03-19T23:11:42.869414+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'8","last_scrub_stamp":"2020-03-19T23:11:42.869407+0000","last_deep_scrub":"24'8","last_deep_scrub_stamp":"2020-03-19T23:11:42.869407+0000","last_clean_scrub_stamp":"2020-03-19T23:11:42.869407+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:39.387377+0000","last_change":"2020-03-19T23:11:39.387432+0000","last_active":"2020-03-19T23:11:39.387377+0000","last_peered":"2020-03-19T23:11:39.387377+0000","last_clean":"2020-03-19T23:11:39.387377+0000","last_became_active":"2020-03-19T23:11:04.651761+0000","last_became_peered":"2020-03-19T23:11:04.651761+0000","last_unstale":"2020-03-19T23:11:39.387377+0000","last_undegraded":"2020-03-19T23:11:39.387377+0000","last_fullsized":"2020-03-19T23:11:39.387377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:39.387371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:39.387371+0000","last_clean_scrub_stamp":"2020-03-19T23:11:39.387371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:30.732884+0000","last_change":"2020-03-19T23:11:30.732947+0000","last_active":"2020-03-19T23:11:30.732884+0000","last_peered":"2020-03-19T23:11:30.732884+0000","last_clean":"2020-03-19T23:11:30.732884+0000","last_became_active":"2020-03-19T23:11:01.250171+0000","last_became_peered":"2020-03-19T23:11:01.250171+0000","last_unstale":"2020-03-19T23:11:30.732884+0000","last_undegraded":"2020-03-19T23:11:30.732884+0000","last_fullsized":"2020-03-19T23:11:30.732884+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:30.732878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:30.732878+0000","last_clean_scrub_stamp":"2020-03-19T23:11:30.732878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":"20","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:36.410635+0000","last_change":"2020-03-19T23:11:36.410689+0000","last_active":"2020-03-19T23:11:36.410635+0000","last_peered":"2020-03-19T23:11:36.410635+0000","last_clean":"2020-03-19T23:11:36.410635+0000","last_became_active":"2020-03-19T23:11:04.287457+0000","last_became_peered":"2020-03-19T23:11:04.287457+0000","last_unstale":"2020-03-19T23:11:36.410635+0000","last_undegraded":"2020-03-19T23:11:36.410635+0000","last_fullsized":"2020-03-19T23:11:36.410635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'1","last_scrub_stamp":"2020-03-19T23:11:36.410629+0000","last_deep_scrub":"16'1","last_deep_scrub_stamp":"2020-03-19T23:11:36.410629+0000","last_clean_scrub_stamp":"2020-03-19T23:11:36.410629+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"15","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:52.338662+0000","last_change":"2020-03-19T23:11:52.338734+0000","last_active":"2020-03-19T23:11:52.338662+0000","last_peered":"2020-03-19T23:11:52.338662+0000","last_clean":"2020-03-19T23:11:52.338662+0000","last_became_active":"2020-03-19T23:11:16.352787+0000","last_became_peered":"2020-03-19T23:11:16.352787+0000","last_unstale":"2020-03-19T23:11:52.338662+0000","last_undegraded":"2020-03-19T23:11:52.338662+0000","last_fullsized":"2020-03-19T23:11:52.338662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:52.338656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:52.338656+0000","last_clean_scrub_stamp":"2020-03-19T23:11:52.338656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"19","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:12:02.919815+0000","last_change":"2020-03-19T23:12:02.919893+0000","last_active":"2020-03-19T23:12:02.919815+0000","last_peered":"2020-03-19T23:12:02.919815+0000","last_clean":"2020-03-19T23:12:02.919815+0000","last_became_active":"2020-03-19T23:11:04.654078+0000","last_became_peered":"2020-03-19T23:11:04.654078+0000","last_unstale":"2020-03-19T23:12:02.919815+0000","last_undegraded":"2020-03-19T23:12:02.919815+0000","last_fullsized":"2020-03-19T23:12:02.919815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:12:02.919808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:12:02.919808+0000","last_clean_scrub_stamp":"2020-03-19T23:12:02.919808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:38.874554+0000","last_change":"2020-03-19T23:11:38.874610+0000","last_active":"2020-03-19T23:11:38.874554+0000","last_peered":"2020-03-19T23:11:38.874554+0000","last_clean":"2020-03-19T23:11:38.874554+0000","last_became_active":"2020-03-19T23:11:01.252205+0000","last_became_peered":"2020-03-19T23:11:01.252205+0000","last_unstale":"2020-03-19T23:11:38.874554+0000","last_undegraded":"2020-03-19T23:11:38.874554+0000","last_fullsized":"2020-03-19T23:11:38.874554+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:38.874549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:38.874549+0000","last_clean_scrub_stamp":"2020-03-19T23:11:38.874549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'10","reported_seq":"30","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:51.836802+0000","last_change":"2020-03-19T23:11:51.836874+0000","last_active":"2020-03-19T23:11:51.836802+0000","last_peered":"2020-03-19T23:11:51.836802+0000","last_clean":"2020-03-19T23:11:51.836802+0000","last_became_active":"2020-03-19T23:11:04.284579+0000","last_became_peered":"2020-03-19T23:11:04.284579+0000","last_unstale":"2020-03-19T23:11:51.836802+0000","last_undegraded":"2020-03-19T23:11:51.836802+0000","last_fullsized":"2020-03-19T23:11:51.836802+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'10","last_scrub_stamp":"2020-03-19T23:11:51.836791+0000","last_deep_scrub":"24'10","last_deep_scrub_stamp":"2020-03-19T23:11:51.836791+0000","last_clean_scrub_stamp":"2020-03-19T23:11:51.836791+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"16","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:12:04.909067+0000","last_change":"2020-03-19T23:12:04.909148+0000","last_active":"2020-03-19T23:12:04.909067+0000","last_peered":"2020-03-19T23:12:04.909067+0000","last_clean":"2020-03-19T23:12:04.909067+0000","last_became_active":"2020-03-19T23:11:16.354173+0000","last_became_peered":"2020-03-19T23:11:16.354173+0000","last_unstale":"2020-03-19T23:12:04.909067+0000","last_undegraded":"2020-03-19T23:12:04.909067+0000","last_fullsized":"2020-03-19T23:12:04.909067+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:12:04.909058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:12:04.909058+0000","last_clean_scrub_stamp":"2020-03-19T23:12:04.909058+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"18","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:42.745317+0000","last_change":"2020-03-19T23:11:42.745375+0000","last_active":"2020-03-19T23:11:42.745317+0000","last_peered":"2020-03-19T23:11:42.745317+0000","last_clean":"2020-03-19T23:11:42.745317+0000","last_became_active":"2020-03-19T23:11:04.650124+0000","last_became_peered":"2020-03-19T23:11:04.650124+0000","last_unstale":"2020-03-19T23:11:42.745317+0000","last_undegraded":"2020-03-19T23:11:42.745317+0000","last_fullsized":"2020-03-19T23:11:42.745317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:42.745302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:42.745302+0000","last_clean_scrub_stamp":"2020-03-19T23:11:42.745302+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"22","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:41.826931+0000","last_change":"2020-03-19T23:11:41.826984+0000","last_active":"2020-03-19T23:11:41.826931+0000","last_peered":"2020-03-19T23:11:41.826931+0000","last_clean":"2020-03-19T23:11:41.826931+0000","last_became_active":"2020-03-19T23:11:01.249024+0000","last_became_peered":"2020-03-19T23:11:01.249024+0000","last_unstale":"2020-03-19T23:11:41.826931+0000","last_undegraded":"2020-03-19T23:11:41.826931+0000","last_fullsized":"2020-03-19T23:11:41.826931+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-03-19T23:11:41.826925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-03-19T23:11:41.826925+0000","last_clean_scrub_stamp":"2020-03-19T23:11:41.826925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'13","reported_seq":"33","reported_epoch":"24","state":"active+clean","last_fresh":"2020-03-19T23:11:46.907519+0000","last_change":"2020-03-19T23:11:46.907579+0000","last_active":"2020-03-19T23:11:46.907519+0000","last_peered":"2020-03-19T23:11:46.907519+0000","last_clean":"2020-03-19T23:11:46.907519+0000","last_became_active":"2020-03-19T23:11:04.283065+0000","last_became_peered":"2020-03-19T23:11:04.283065+0000","last_unstale":"2020-03-19T23:11:46.907519+0000","last_undegraded":"2020-03-19T23:11:46.907519+0000","last_fullsized":"2020-03-19T23:11:46.907519+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'13","last_scrub_stamp":"2020-03-19T23:11:46.907513+0000","last_deep_scrub":"24'13","last_deep_scrub_stamp":"2020-03-19T23:11:46.907513+0000","last_clean_scrub_stamp":"2020-03-19T23:11:46.907513+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1958,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":3},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":2718,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4620,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61,"ondisk_log_size":61,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738385,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048788,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808812,"statfs":{"total":107374182400,"available":106300223488,"internally_reserved":1073741824,"allocated":208896,"data_stored":77785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 19 23:12:01 2020","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.83399999999999996}]},{"osd":1,"last update":"Thu Mar 19 23:12:01 2020","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.879}]}]},{"osd":0,"up_from":8,"seq":34359738384,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808788,"statfs":{"total":107374182400,"available":106300198912,"internally_reserved":1073741824,"allocated":233472,"data_stored":80435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 19 23:11:59 2020","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Mar 19 23:11:59 2020","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.747}]}]},{"osd":1,"up_from":8,"seq":34359738384,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":103808784,"statfs":{"total":107374182400,"available":106300194816,"internally_reserved":1073741824,"allocated":237568,"data_stored":80469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 19 23:11:56 2020","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Thu Mar 19 23:11:56 2020","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.746},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.878}]}]}]}} 2020-03-19T23:12:10.130 INFO:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2020-03-19T23:12:10.253 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff559619700 1 Processor -- start 2020-03-19T23:12:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff559619700 1 -- start start 2020-03-19T23:12:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff559619700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554137cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect 2020-03-19T23:12:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff559619700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540902c0 con 0x7ff55408f010 2020-03-19T23:12:10.254 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff559619700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540904f0 con 0x7ff55408fb00 2020-03-19T23:12:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff54affd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554137cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:12:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff5537fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff55408f010 0x7ff55408f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44402/0 (socket says 172.21.15.38:44402) 2020-03-19T23:12:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.244+0000 7ff5537fe700 1 -- 172.21.15.38:0/76444311 learned_addr learned my addr 172.21.15.38:0/76444311 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:12:10.255 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 -- 172.21.15.38:0/76444311 >> v1:172.21.15.38:6789/0 conn(0x7ff55408f010 legacy=0x7ff55408f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 -- 172.21.15.38:0/76444311 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540954e0 con 0x7ff55408fb00 2020-03-19T23:12:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff552ffd700 1 -- 172.21.15.38:0/76444311 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164632353 0 0) 0x7ff5540902c0 con 0x7ff55408f010 2020-03-19T23:12:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 --2- 172.21.15.38:0/76444311 >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554137cf0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.256 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 --2- 172.21.15.38:0/76444311 >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554137cf0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 --2- 172.21.15.38:0/76444311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554137cf0 secure :-1 s=READY pgs=307 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).ready entity=mon.0 client_cookie=65576a4786a8ff5d server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:12:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 --2- 172.21.15.38:0/76444311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554137cf0 secure :-1 s=THROTTLE_DONE pgs=307 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff552ffd700 1 -- 172.21.15.38:0/76444311 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff53c008e80 con 0x7ff55408fb00 2020-03-19T23:12:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 --2- 172.21.15.38:0/76444311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554137cf0 secure :-1 s=THROTTLE_DONE pgs=307 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff552ffd700 1 -- 172.21.15.38:0/76444311 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c0113f0 con 0x7ff55408fb00 2020-03-19T23:12:10.258 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff54affd700 1 --2- 172.21.15.38:0/76444311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554137cf0 secure :-1 s=THROTTLE_DONE pgs=307 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff552ffd700 1 -- 172.21.15.38:0/76444311 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff53c0116d0 con 0x7ff55408fb00 2020-03-19T23:12:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff559619700 1 -- 172.21.15.38:0/76444311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 msgr2=0x7ff554137cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff559619700 1 --2- 172.21.15.38:0/76444311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554137cf0 secure :-1 s=READY pgs=307 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).stop 2020-03-19T23:12:10.259 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff559619700 1 -- 172.21.15.38:0/76444311 shutdown_connections 2020-03-19T23:12:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.245+0000 7ff559619700 1 --2- 172.21.15.38:0/76444311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554137cf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:12:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.246+0000 7ff559619700 1 -- 172.21.15.38:0/76444311 >> 172.21.15.38:0/76444311 conn(0x7ff554095090 msgr2=0x7ff554091110 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:12:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.246+0000 7ff559619700 1 -- 172.21.15.38:0/76444311 shutdown_connections 2020-03-19T23:12:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.246+0000 7ff559619700 1 -- 172.21.15.38:0/76444311 wait complete. 2020-03-19T23:12:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.246+0000 7ff559619700 1 Processor -- start 2020-03-19T23:12:10.261 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff559619700 1 -- start start 2020-03-19T23:12:10.261 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff559619700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:12:10.261 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff559619700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff554072ec0 con 0x7ff55408f010 2020-03-19T23:12:10.261 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff559619700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55407daf0 con 0x7ff55408fb00 2020-03-19T23:12:10.262 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff54affd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:12:10.262 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff5537fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff55408f010 0x7ff554071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44408/0 (socket says 172.21.15.38:44408) 2020-03-19T23:12:10.262 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff54affd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554074120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33034/0 (socket says 172.21.15.38:33034) 2020-03-19T23:12:10.262 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff5537fe700 1 -- 172.21.15.38:0/950657574 learned_addr learned my addr 172.21.15.38:0/950657574 (peer_addr_for_me v1:172.21.15.38:0/0) 2020-03-19T23:12:10.263 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff54affd700 1 -- 172.21.15.38:0/950657574 >> v1:172.21.15.38:6789/0 conn(0x7ff55408f010 legacy=0x7ff554071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:10.263 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff54affd700 1 -- 172.21.15.38:0/950657574 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55407d5c0 con 0x7ff55408fb00 2020-03-19T23:12:10.263 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819400267 0 0) 0x7ff554072ec0 con 0x7ff55408f010 2020-03-19T23:12:10.263 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.263 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> v2:172.21.15.38:3300/0 conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.247+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=READY pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:12:10.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff53c0116c0 con 0x7ff55408fb00 2020-03-19T23:12:10.264 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c00f830 con 0x7ff55408fb00 2020-03-19T23:12:10.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff554074580 con 0x7ff55408fb00 2020-03-19T23:12:10.265 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 192+0+0 (secure 0 0 0) 0x7ff53c007280 con 0x7ff55408fb00 2020-03-19T23:12:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff554072a40 con 0x7ff55408fb00 2020-03-19T23:12:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 29603+0+0 (secure 0 0 0) 0x7ff53c0074b0 con 0x7ff55408fb00 2020-03-19T23:12:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff550ff9700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 0x7ff53802a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rx=0 tx=0).connect 2020-03-19T23:12:10.267 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.248+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3786+0+0 (secure 0 0 0) 0x7ff53c034910 con 0x7ff55408fb00 2020-03-19T23:12:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.249+0000 7ff5537fe700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 0x7ff53802a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0 2020-03-19T23:12:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.249+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554056db0 con 0x7ff55408fb00 2020-03-19T23:12:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.249+0000 7ff5537fe700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 0x7ff53802a740 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rx=0x7ff544005500 tx=0x7ff544001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.268 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.249+0000 7ff5537fe700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 0x7ff53802a740 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rx=0x7ff544005500 tx=0x7ff544001700).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.249+0000 7ff5537fe700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 0x7ff53802a740 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7ff544005500 tx=0x7ff544001700).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-03-19T23:12:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.252+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.252+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+126772 (secure 0 0 0) 0x7ff53c009870 con 0x7ff55408fb00 2020-03-19T23:12:10.402 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.400+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"} v 0) v1 -- 0x7ff5540008d0 con 0x7ff55408fb00 2020-03-19T23:12:10.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.408+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.408+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff53c009e70 con 0x7ff55408fb00 2020-03-19T23:12:10.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.408+0000 7ff54affd700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=THROTTLE_DONE pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).handle_read_frame_epilogue_main read frame epilogue bytes=32 2020-03-19T23:12:10.410 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.408+0000 7ff550ff9700 1 -- 172.21.15.38:0/950657574 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}]=0 v3) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff53c009870 con 0x7ff55408fb00 2020-03-19T23:12:10.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 msgr2=0x7ff53802a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:10.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 0x7ff53802a740 secure :-1 s=READY pgs=77 cs=0 l=1 rx=0x7ff544005500 tx=0x7ff544001700).stop 2020-03-19T23:12:10.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 msgr2=0x7ff554074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-03-19T23:12:10.412 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 secure :-1 s=READY pgs=309 cs=0 l=1 rx=0x7ff53c002f30 tx=0x7ff53c007a40).stop 2020-03-19T23:12:10.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 shutdown_connections 2020-03-19T23:12:10.413 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:6824/20200,v1:172.21.15.38:6825/20200] conn(0x7ff538028280 0x7ff53802a740 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:12:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 --2- 172.21.15.38:0/950657574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff55408fb00 0x7ff554074120 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rx=0 tx=0).stop 2020-03-19T23:12:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.410+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 >> 172.21.15.38:0/950657574 conn(0x7ff554095090 msgr2=0x7ff554066610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-03-19T23:12:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.411+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 shutdown_connections 2020-03-19T23:12:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:2020-03-19T23:12:10.411+0000 7ff559619700 1 -- 172.21.15.38:0/950657574 wait complete. 2020-03-19T23:12:10.423 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph.py", line 1892, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph_fuse.py", line 140, in task info["mount"].mount(mountpoint=mountpoint, mount_path=mount_path) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 33, in mount return self._mount(mount_path, mount_fs_name, mount_options) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 163, in _mount self.gather_mount_info() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 166, in gather_mount_info status = self.admin_socket(['status']) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 449, in admin_socket ], stdout=StringIO(), timeout=(15*60)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi038 with status 1: 'sudo python3 -c \'\'"\'"\'\nimport glob\nimport re\nimport os\nimport subprocess\n\ndef find_socket(client_name):\n asok_path = "/var/run/ceph/ceph-client.1.*.asok"\n files = glob.glob(asok_path)\n\n # Given a non-glob path, it better be there\n if "*" not in asok_path:\n assert(len(files) == 1)\n return files[0]\n\n for f in files:\n pid = re.match(".*\\.(\\d+)\\.asok$", f).group(1)\n if os.path.exists("/proc/{0}".format(pid)):\n return f\n raise RuntimeError("Client socket {0} not found".format(client_name))\n\nprint(find_socket("client.1"))\n\'"\'"\'\'' 2020-03-19T23:12:10.423 INFO:teuthology.misc:Shutting down mds daemons... 2020-03-19T23:12:10.424 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-03-19T23:12:10.424 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T23:12:10.432 INFO:tasks.ceph.mds.a:Stopped 2020-03-19T23:12:10.432 INFO:teuthology.misc:Shutting down osd daemons... 2020-03-19T23:12:10.432 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2020-03-19T23:12:10.432 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T23:12:10.498 INFO:tasks.ceph.osd.1:Stopped 2020-03-19T23:12:10.498 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2020-03-19T23:12:10.498 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T23:12:10.570 INFO:tasks.ceph.osd.0:Stopped 2020-03-19T23:12:10.570 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2020-03-19T23:12:10.570 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T23:12:10.645 INFO:tasks.ceph.osd.2:Stopped 2020-03-19T23:12:10.646 INFO:teuthology.misc:Shutting down mgr daemons... 2020-03-19T23:12:10.646 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2020-03-19T23:12:10.646 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T23:12:10.702 INFO:tasks.ceph.mgr.x:Stopped 2020-03-19T23:12:10.703 INFO:teuthology.misc:Shutting down mon daemons... 2020-03-19T23:12:10.703 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2020-03-19T23:12:10.703 INFO:teuthology.orchestra.run:waiting for 300 2020-03-19T23:12:10.752 INFO:tasks.ceph.mon.a:Stopped 2020-03-19T23:12:10.753 INFO:tasks.ceph:Checking cluster log for badness... 2020-03-19T23:12:10.753 INFO:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2020-03-19T23:12:10.826 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:12:10.826 INFO:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2020-03-19T23:12:11.087 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:12:11.087 INFO:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2020-03-19T23:12:11.196 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:12:11.196 INFO:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2020-03-19T23:12:11.276 INFO:tasks.ceph:Archiving mon data... 2020-03-19T23:12:11.277 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731/data/mon.a.tgz 2020-03-19T23:12:11.277 INFO:teuthology.orchestra.run.smithi038:> mktemp 2020-03-19T23:12:11.291 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.cUclhVJ6Z3 2020-03-19T23:12:11.292 INFO:teuthology.orchestra.run.smithi038:> sudo tar cz -f /tmp/tmp.cUclhVJ6Z3 -C /var/lib/ceph/mon/ceph-a -- . 2020-03-19T23:12:11.426 INFO:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.cUclhVJ6Z3 2020-03-19T23:12:11.514 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.cUclhVJ6Z3 is 264KB 2020-03-19T23:12:11.611 INFO:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.cUclhVJ6Z3 2020-03-19T23:12:11.626 INFO:tasks.ceph:Cleaning ceph cluster... 2020-03-19T23:12:11.626 INFO:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2020-03-19T23:12:11.695 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2020-03-19T23:12:11.695 INFO:teuthology.orchestra.run.smithi038:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2020-03-19T23:12:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2020-03-19T23:12:11.773 INFO:tasks.ceph:Archiving crash dumps... 2020-03-19T23:12:11.774 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/crash to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731/remote/smithi038/crash 2020-03-19T23:12:11.775 INFO:teuthology.orchestra.run.smithi038:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2020-03-19T23:12:11.845 INFO:tasks.ceph:Compressing logs... 2020-03-19T23:12:11.846 INFO:teuthology.orchestra.run.smithi038:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-03-19T23:12:12.356 INFO:tasks.ceph:Archiving logs... 2020-03-19T23:12:12.357 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731/remote/smithi038/log 2020-03-19T23:12:12.358 INFO:teuthology.orchestra.run.smithi038:> sudo tar cz -f - -C /var/log/ceph -- . 2020-03-19T23:12:12.546 DEBUG:teuthology.run_tasks:Unwinding manager install 2020-03-19T23:12:12.571 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/install/__init__.py", line 599, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/ceph_fuse.py", line 140, in task info["mount"].mount(mountpoint=mountpoint, mount_path=mount_path) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 33, in mount return self._mount(mount_path, mount_fs_name, mount_options) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 163, in _mount self.gather_mount_info() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 166, in gather_mount_info status = self.admin_socket(['status']) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-sage-testing-2020-03-19-0855/qa/tasks/cephfs/fuse_mount.py", line 449, in admin_socket ], stdout=StringIO(), timeout=(15*60)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi038 with status 1: 'sudo python3 -c \'\'"\'"\'\nimport glob\nimport re\nimport os\nimport subprocess\n\ndef find_socket(client_name):\n asok_path = "/var/run/ceph/ceph-client.1.*.asok"\n files = glob.glob(asok_path)\n\n # Given a non-glob path, it better be there\n if "*" not in asok_path:\n assert(len(files) == 1)\n return files[0]\n\n for f in files:\n pid = re.match(".*\\.(\\d+)\\.asok$", f).group(1)\n if os.path.exists("/proc/{0}".format(pid)):\n return f\n raise RuntimeError("Client socket {0} not found".format(client_name))\n\nprint(find_socket("client.1"))\n\'"\'"\'\'' 2020-03-19T23:12:12.572 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2020-03-19T23:12:12.572 INFO:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2020-03-19T23:12:12.604 INFO:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2020-03-19T23:12:12.686 INFO:teuthology.orchestra.run.smithi038.stdout:RedHatEnterprise 2020-03-19T23:12:12.687 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T23:12:12.687 INFO:teuthology.task.install.rpm:Removing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on rpm system. 2020-03-19T23:12:12.687 INFO:teuthology.orchestra.run.smithi038:> 2020-03-19T23:12:12.688 INFO:teuthology.orchestra.run.smithi038:> for d in ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ; do 2020-03-19T23:12:12.688 INFO:teuthology.orchestra.run.smithi038:> sudo yum -y remove $d || true 2020-03-19T23:12:12.688 INFO:teuthology.orchestra.run.smithi038:> done 2020-03-19T23:12:13.104 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:12:13.730 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:12:13.731 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:12:13.731 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:12:13.731 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:12:13.770 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:12:13.771 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:13.771 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2020-03-19T23:12:13.771 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:13.771 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:12:13.771 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:15.1.1-234.g787db24.el8 @ceph 40 M 2020-03-19T23:12:13.772 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:13.772 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:12:13.772 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:13.772 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 1 Package 2020-03-19T23:12:13.772 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:13.772 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 40 M 2020-03-19T23:12:13.773 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:12:13.774 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:12:13.775 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:12:13.805 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:12:13.805 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:12:13.870 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:12:13.903 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:12:13.904 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:12:14.451 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:12:16.787 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:12:16.787 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:12:16.985 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:16.986 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:12:16.986 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:12:16.986 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:16.986 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:12:17.530 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:12:18.147 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:12:18.147 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:12:18.148 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:12:18.148 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:12:18.187 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:12:18.188 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2020-03-19T23:12:18.189 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:18.189 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:12:18.189 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:15.1.1-234.g787db24.el8 2020-03-19T23:12:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: @ceph 182 M 2020-03-19T23:12:18.190 INFO:teuthology.orchestra.run.smithi038.stdout:Removing unused dependencies: 2020-03-19T23:12:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: jq x86_64 1.5-12.el8 @rhel-8-for-x86_64-appstream-rpms 357 k 2020-03-19T23:12:18.190 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma x86_64 6.8.2-1.el8 @rhel-8-for-x86_64-appstream-rpms 692 k 2020-03-19T23:12:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.3.2-6.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M 2020-03-19T23:12:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-11.el8 @epel 258 k 2020-03-19T23:12:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:18.191 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:12:18.191 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:18.191 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 5 Packages 2020-03-19T23:12:18.191 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:18.192 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 184 M 2020-03-19T23:12:18.192 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:12:18.194 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:12:18.194 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:12:18.245 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:12:18.245 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:12:18.383 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:12:18.588 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 1/5 2020-03-19T23:12:18.763 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : jq-1.5-12.el8.x86_64 2/5 2020-03-19T23:12:18.764 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : oniguruma-6.8.2-1.el8.x86_64 3/5 2020-03-19T23:12:18.988 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 3/5 2020-03-19T23:12:19.139 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : socat-1.7.3.2-6.el8.x86_64 4/5 2020-03-19T23:12:19.325 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : xmlstarlet-1.6.1-11.el8.x86_64 5/5 2020-03-19T23:12:19.554 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: xmlstarlet-1.6.1-11.el8.x86_64 5/5 2020-03-19T23:12:19.554 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:15.1.1-234.g787db24.el8.x86_64 1/5 2020-03-19T23:12:19.555 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : jq-1.5-12.el8.x86_64 2/5 2020-03-19T23:12:19.555 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 3/5 2020-03-19T23:12:19.555 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.3.2-6.el8.x86_64 4/5 2020-03-19T23:12:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 5/5 2020-03-19T23:12:21.852 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:12:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:22.045 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:12:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:15.1.1-234.g787db24.el8.x86_64 jq-1.5-12.el8.x86_64 2020-03-19T23:12:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma-6.8.2-1.el8.x86_64 socat-1.7.3.2-6.el8.x86_64 2020-03-19T23:12:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-03-19T23:12:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:22.046 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:12:22.605 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:12:23.243 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:12:23.244 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:12:23.244 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:12:23.244 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:12:23.283 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:12:23.286 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2020-03-19T23:12:23.286 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:23.286 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:12:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:15.1.1-234.g787db24.el8 @ceph 0 2020-03-19T23:12:23.287 INFO:teuthology.orchestra.run.smithi038.stdout:Removing unused dependencies: 2020-03-19T23:12:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:15.1.1-234.g787db24.el8 @ceph 7.3 M 2020-03-19T23:12:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:15.1.1-234.g787db24.el8 @ceph 15 M 2020-03-19T23:12:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:15.1.1-234.g787db24.el8 @ceph 59 M 2020-03-19T23:12:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:23.288 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:12:23.288 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:23.288 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 4 Packages 2020-03-19T23:12:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:23.289 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 81 M 2020-03-19T23:12:23.289 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:12:23.290 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:12:23.290 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:12:23.319 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:12:23.320 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:12:23.402 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:12:23.409 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:12:23.571 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-2:15.1.1-234.g787db24.el8.x86_64 1/4 2020-03-19T23:12:23.591 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T23:12:23.592 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T23:12:23.880 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T23:12:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T23:12:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T23:12:24.054 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T23:12:24.083 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T23:12:24.084 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T23:12:24.469 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T23:12:24.470 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:15.1.1-234.g787db24.el8.x86_64 1/4 2020-03-19T23:12:24.470 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2/4 2020-03-19T23:12:24.470 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 3/4 2020-03-19T23:12:26.884 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 4/4 2020-03-19T23:12:26.884 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:12:27.085 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:27.085 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:12:27.085 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:12:27.085 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:12:27.086 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:12:27.086 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:12:27.086 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:27.086 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:12:27.656 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:12:28.352 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:12:28.352 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:12:28.352 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:12:28.352 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:12:28.401 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:12:28.417 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:28.417 INFO:teuthology.orchestra.run.smithi038.stdout: Package ArchVersion Repository Size 2020-03-19T23:12:28.418 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:28.418 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:12:28.418 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_642:15.1.1-234.g787db24.el8 @ceph 26 M 2020-03-19T23:12:28.418 INFO:teuthology.orchestra.run.smithi038.stdout:Removing dependent packages: 2020-03-19T23:12:28.418 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 107 k 2020-03-19T23:12:28.418 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_642:15.1.1-234.g787db24.el8 @ceph 13 M 2020-03-19T23:12:28.419 INFO:teuthology.orchestra.run.smithi038.stdout:Removing unused dependencies: 2020-03-19T23:12:28.419 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_642:15.1.1-234.g787db24.el8 @ceph 79 M 2020-03-19T23:12:28.419 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 201 k 2020-03-19T23:12:28.419 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_642:15.1.1-234.g787db24.el8 @ceph 5.3 M 2020-03-19T23:12:28.419 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 142 k 2020-03-19T23:12:28.419 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 17 M 2020-03-19T23:12:28.419 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 66 M 2020-03-19T23:12:28.420 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch2:15.1.1-234.g787db24.el8 @ceph-noarch 872 k 2020-03-19T23:12:28.420 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_642:15.1.1-234.g787db24.el8 @ceph 131 k 2020-03-19T23:12:28.420 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_642.7-6.el8 @epel 1.7 M 2020-03-19T23:12:28.420 INFO:teuthology.orchestra.run.smithi038.stdout: leveldb x86_641.21-2.el8 @lab-extras 482 k 2020-03-19T23:12:28.420 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_648.3.1-4.5.el8 @rhel-8-for-x86_64-baseos-rpms 2.5 M 2020-03-19T23:12:28.420 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_642.6.2-3.el8 @epel 125 k 2020-03-19T23:12:28.420 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_648.3.1-4.5.el8 @rhel-8-for-x86_64-baseos-rpms 299 k 2020-03-19T23:12:28.421 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_640.9.0-1.el8 @rhel-8-for-x86_64-baseos-rpms 95 k 2020-03-19T23:12:28.421 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_642:15.1.1-234.g787db24.el8 @ceph 1.4 M 2020-03-19T23:12:28.421 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_640.11.4-1.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M 2020-03-19T23:12:28.421 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_642:15.1.1-234.g787db24.el8 @ceph 25 M 2020-03-19T23:12:28.421 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_641.3.1-3.el8 @epel 205 k 2020-03-19T23:12:28.421 INFO:teuthology.orchestra.run.smithi038.stdout: libxslt x86_641.1.32-3.el8 @rhel-8-for-x86_64-baseos-rpms 743 k 2020-03-19T23:12:28.421 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_640.3.3-2.el8 @rhel-8-for-x86_64-appstream-rpms 31 M 2020-03-19T23:12:28.422 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads x86_640.3.3-2.el8 @rhel-8-for-x86_64-appstream-rpms 32 M 2020-03-19T23:12:28.422 INFO:teuthology.orchestra.run.smithi038.stdout: python3-apipkg noarch1.5-6.el8 @epel 24 k 2020-03-19T23:12:28.422 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_643.1.6-2.el8.1 @epel 78 k 2020-03-19T23:12:28.422 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4 noarch4.6.3-2.el8.1 @epel 817 k 2020-03-19T23:12:28.422 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch3.1.1-4.el8 @epel 71 k 2020-03-19T23:12:28.423 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_642:15.1.1-234.g787db24.el8 @ceph 113 k 2020-03-19T23:12:28.423 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_642:15.1.1-234.g787db24.el8 @ceph 216 k 2020-03-19T23:12:28.423 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_642:15.1.1-234.g787db24.el8 @ceph 520 k 2020-03-19T23:12:28.423 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch2018.10.15-7.el8 @epel 8.2 k 2020-03-19T23:12:28.423 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch8.2.1-1.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 531 k 2020-03-19T23:12:28.423 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch18.4.0-1.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 1.1 M 2020-03-19T23:12:28.424 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cssselect noarch0.9.2-10.el8 @epel 142 k 2020-03-19T23:12:28.424 INFO:teuthology.orchestra.run.smithi038.stdout: python3-execnet noarch1.7.1-1.el8.2 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 1.2 M 2020-03-19T23:12:28.424 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch1:1.1.1-10.el8 @epel 306 k 2020-03-19T23:12:28.424 INFO:teuthology.orchestra.run.smithi038.stdout: python3-html5lib noarch1:0.999999999-6.el8 @rhel-8-for-x86_64-appstream-rpms 1.1 M 2020-03-19T23:12:28.424 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch6.2-6.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 4.0 k 2020-03-19T23:12:28.424 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch2.0-4.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 28 k 2020-03-19T23:12:28.424 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch1.21-2.el8 @rhel-8-for-x86_64-appstream-rpms 55 k 2020-03-19T23:12:28.425 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch1.10-11.el8 @rhel-8-for-x86_64-appstream-rpms 32 k 2020-03-19T23:12:28.425 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt noarch1.6.1-2.el8 @rhel-8-for-x86_64-baseos-rpms 88 k 2020-03-19T23:12:28.425 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch10.0.1-2.el8 @epel 20 M 2020-03-19T23:12:28.425 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch0.3.5-11.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 126 k 2020-03-19T23:12:28.425 INFO:teuthology.orchestra.run.smithi038.stdout: python3-lxml x86_644.2.3-1.el8 @rhel-8-for-x86_64-appstream-rpms 4.8 M 2020-03-19T23:12:28.425 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch1.0.6-13.el8 @rhel-8-for-x86_64-appstream-rpms 540 k 2020-03-19T23:12:28.426 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_640.23-19.el8 @rhel-8-for-x86_64-appstream-rpms 76 k 2020-03-19T23:12:28.426 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch7.2.0-3.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 302 k 2020-03-19T23:12:28.426 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_641:1.14.3-9.el8 @rhel-8-for-x86_64-appstream-rpms 16 M 2020-03-19T23:12:28.426 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_641:1.14.3-9.el8 @rhel-8-for-x86_64-appstream-rpms 946 k 2020-03-19T23:12:28.426 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch2.1.0-1.el8 @rhel-8-for-x86_64-baseos-rpms 737 k 2020-03-19T23:12:28.426 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch1.3.2-9.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 1.3 M 2020-03-19T23:12:28.426 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch2.6-1.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 19 k 2020-03-19T23:12:28.427 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch0.7.2-14.el8 @rhel-8-for-x86_64-appstream-rpms 167 k 2020-03-19T23:12:28.427 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch0.3.7-6.el8 @rhel-8-for-x86_64-appstream-rpms 522 k 2020-03-19T23:12:28.427 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch0.3.7-6.el8 @rhel-8-for-x86_64-appstream-rpms 603 k 2020-03-19T23:12:28.427 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch2017.2-9.el8 @rhel-8-for-x86_64-appstream-rpms 175 k 2020-03-19T23:12:28.427 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_642:15.1.1-234.g787db24.el8 @ceph 1.1 M 2020-03-19T23:12:28.427 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_642:15.1.1-234.g787db24.el8 @ceph 1.1 M 2020-03-19T23:12:28.427 INFO:teuthology.orchestra.run.smithi038.stdout: python3-remoto noarch1.1.4-4.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 97 k 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch0.7-6.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 84 k 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch1.0.0-1.el8 @rhel-8-for-x86_64-baseos-rpms 107 k 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_642:15.1.1-234.g787db24.el8 @ceph 293 k 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch2.4.1-12.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 454 k 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch3.4.2-12.el8 @epel 240 k 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_641.0.0-20.module+el8.1.0+3334+5cb623d7 @rhel-8-for-x86_64-appstream-rpms 56 M 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric noarch0.8.1-17.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 24 k 2020-03-19T23:12:28.428 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch noarch3.4.0.3-18.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 43 k 2020-03-19T23:12:28.429 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch1.14.1-5.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 82 k 2020-03-19T23:12:28.429 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme noarch0.5.2-4.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 51 k 2020-03-19T23:12:28.429 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress noarch1.2.1-2.el8.1 @epel 1.0 M 2020-03-19T23:12:28.429 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webencodings noarch0.5.1-6.el8 @rhel-8-for-x86_64-appstream-rpms 72 k 2020-03-19T23:12:28.429 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch1.8.5-1.el8.1 @epel 1.2 M 2020-03-19T23:12:28.430 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch0.56.0-5.el8 @epel 180 k 2020-03-19T23:12:28.430 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest noarch2.0.33-1.el8 @epel 289 k 2020-03-19T23:12:28.430 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch0.12.2-4.el8 @rhel-8-for-x86_64-appstream-rpms 2.0 M 2020-03-19T23:12:28.430 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch2.0-2.el8 @copr:copr.fedorainfracloud.org:ktdreyer:ceph-el8 35 k 2020-03-19T23:12:28.430 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:28.430 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:12:28.430 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:12:28.431 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 78 Packages 2020-03-19T23:12:28.431 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:28.431 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 422 M 2020-03-19T23:12:28.431 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:12:28.455 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:12:28.455 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:12:28.548 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:12:28.548 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:12:29.600 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:12:29.629 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/1 2020-03-19T23:12:29.629 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/78 2020-03-19T23:12:29.787 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 1/78 2020-03-19T23:12:29.806 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T23:12:29.807 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T23:12:30.083 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T23:12:30.298 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 3/78 2020-03-19T23:12:30.604 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-pecan-1.3.2-9.el8.noarch 4/78 2020-03-19T23:12:30.605 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarc 5/78 2020-03-19T23:12:30.962 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarc 5/78 2020-03-19T23:12:31.275 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-cherrypy-18.4.0-1.el8.noarch 6/78 2020-03-19T23:12:31.542 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-kubernetes-10.0.1-2.el8.noarch 7/78 2020-03-19T23:12:31.758 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-google-auth-1:1.1.1-10.el8.noarch 8/78 2020-03-19T23:12:31.931 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-cheroot-8.2.1-1.el8.noarch 9/78 2020-03-19T23:12:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-webtest-2.0.33-1.el8.noarch 10/78 2020-03-19T23:12:32.174 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db2 11/78 2020-03-19T23:12:32.482 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db2 11/78 2020-03-19T23:12:32.762 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d 12/78 2020-03-19T23:12:32.788 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 13/78 2020-03-19T23:12:32.788 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 13/78 2020-03-19T23:12:32.995 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 13/78 2020-03-19T23:12:32.996 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 14/78 2020-03-19T23:12:33.199 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 14/78 2020-03-19T23:12:33.399 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 15/78 2020-03-19T23:12:33.525 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-pyasn1-modules-0.3.7-6.el8.noarch 16/78 2020-03-19T23:12:33.707 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-rsa-3.4.2-12.el8.noarch 17/78 2020-03-19T23:12:33.897 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-requests-oauthlib-1.0.0-1.el8.noarch 18/78 2020-03-19T23:12:34.046 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-oauthlib-2.1.0-1.el8.noarch 19/78 2020-03-19T23:12:34.230 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-portend-2.6-1.el8.noarch 20/78 2020-03-19T23:12:34.396 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-tempora-1.14.1-5.el8.noarch 21/78 2020-03-19T23:12:34.581 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-jaraco-functools-2.0-4.el8.noarch 22/78 2020-03-19T23:12:34.753 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-routes-2.4.1-12.el8.noarch 23/78 2020-03-19T23:12:34.952 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-mako-1.0.6-13.el8.noarch 24/78 2020-03-19T23:12:34.952 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 25/78 2020-03-19T23:12:35.142 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 25/78 2020-03-19T23:12:35.329 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-remoto-1.1.4-4.el8.noarch 26/78 2020-03-19T23:12:35.538 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-execnet-1.7.1-1.el8.2.noarch 27/78 2020-03-19T23:12:35.720 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-jsonpatch-1.21-2.el8.noarch 28/78 2020-03-19T23:12:35.871 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-jsonpointer-1.10-11.el8.noarch 29/78 2020-03-19T23:12:36.054 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-apipkg-1.5-6.el8.noarch 30/78 2020-03-19T23:12:36.228 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-repoze-lru-0.7-6.el8.noarch 31/78 2020-03-19T23:12:36.396 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-jaraco-6.2-6.el8.noarch 32/78 2020-03-19T23:12:36.531 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-pytz-2017.2-9.el8.noarch 33/78 2020-03-19T23:12:36.736 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-jwt-1.6.1-2.el8.noarch 34/78 2020-03-19T23:12:36.887 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-pyasn1-0.3.7-6.el8.noarch 35/78 2020-03-19T23:12:37.102 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-waitress-1.2.1-2.el8.1.noarch 36/78 2020-03-19T23:12:37.312 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-webob-1.8.5-1.el8.1.noarch 37/78 2020-03-19T23:12:37.471 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-more-itertools-7.2.0-3.el8.noarch 38/78 2020-03-19T23:12:37.664 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-trustme-0.5.2-4.el8.noarch 39/78 2020-03-19T23:12:37.821 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-cachetools-3.1.1-4.el8.noarch 40/78 2020-03-19T23:12:38.014 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-certifi-2018.10.15-7.el8.noarch 41/78 2020-03-19T23:12:38.221 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-websocket-client-0.56.0-5.el8.noarch 42/78 2020-03-19T23:12:38.371 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-zc-lockfile-2.0-2.el8.noarch 43/78 2020-03-19T23:12:38.546 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8. 44/78 2020-03-19T23:12:38.697 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-werkzeug-0.12.2-4.el8.noarch 45/78 2020-03-19T23:12:38.895 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-logutils-0.3.5-11.el8.noarch 46/78 2020-03-19T23:12:39.028 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-simplegeneric-0.8.1-17.el8.noarch 47/78 2020-03-19T23:12:39.204 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-singledispatch-3.4.0.3-18.el8.noarch 48/78 2020-03-19T23:12:39.426 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-lxml-4.2.3-1.el8.x86_64 49/78 2020-03-19T23:12:39.477 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 50/78 2020-03-19T23:12:39.478 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 50/78 2020-03-19T23:12:39.478 INFO:teuthology.orchestra.run.smithi038.stdout:warning: file /etc/logrotate.d/ceph: remove failed: No such file or directory 2020-03-19T23:12:39.478 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:39.822 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 50/78 2020-03-19T23:12:39.823 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 51/78 2020-03-19T23:12:39.823 INFO:teuthology.orchestra.run.smithi038.stdout:warning: directory /var/lib/ceph: remove failed: Device or resource busy 2020-03-19T23:12:39.823 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:40.062 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 51/78 2020-03-19T23:12:40.230 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 52/78 2020-03-19T23:12:40.231 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : librgw2-2:15.1.1-234.g787db24.el8.x86_64 53/78 2020-03-19T23:12:40.487 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:15.1.1-234.g787db24.el8.x86_64 53/78 2020-03-19T23:12:40.723 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 54/78 2020-03-19T23:12:40.926 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-numpy-1:1.14.3-9.el8.x86_64 55/78 2020-03-19T23:12:41.070 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-html5lib-1:0.999999999-6.el8.noarch 56/78 2020-03-19T23:12:41.253 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-webencodings-0.5.1-6.el8.noarch 57/78 2020-03-19T23:12:41.373 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 58/78 2020-03-19T23:12:41.528 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 59/78 2020-03-19T23:12:41.661 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-prettytable-0.7.2-14.el8.noarch 60/78 2020-03-19T23:12:41.661 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 61/78 2020-03-19T23:12:57.247 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 61/78 2020-03-19T23:12:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2020-03-19T23:12:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2020-03-19T23:12:57.248 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2020-03-19T23:12:57.248 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /run 2020-03-19T23:12:57.248 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2020-03-19T23:12:57.248 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2020-03-19T23:12:57.248 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2020-03-19T23:12:57.248 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2020-03-19T23:12:57.248 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2020-03-19T23:12:57.249 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:12:57.539 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-cssselect-0.9.2-10.el8.noarch 62/78 2020-03-19T23:12:57.540 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : openblas-0.3.3-2.el8.x86_64 63/78 2020-03-19T23:12:57.803 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-0.3.3-2.el8.x86_64 63/78 2020-03-19T23:12:57.803 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : openblas-threads-0.3.3-2.el8.x86_64 64/78 2020-03-19T23:12:58.042 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-threads-0.3.3-2.el8.x86_64 64/78 2020-03-19T23:12:58.043 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libgfortran-8.3.1-4.5.el8.x86_64 65/78 2020-03-19T23:12:58.234 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libgfortran-8.3.1-4.5.el8.x86_64 65/78 2020-03-19T23:12:58.410 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : leveldb-1.21-2.el8.x86_64 66/78 2020-03-19T23:12:58.571 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : gperftools-libs-2.7-6.el8.x86_64 67/78 2020-03-19T23:12:58.734 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 68/78 2020-03-19T23:12:58.926 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 69/78 2020-03-19T23:12:59.141 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libunwind-1.3.1-3.el8.x86_64 70/78 2020-03-19T23:12:59.148 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 71/78 2020-03-19T23:12:59.149 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libquadmath-8.3.1-4.5.el8.x86_64 71/78 2020-03-19T23:12:59.359 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libquadmath-8.3.1-4.5.el8.x86_64 71/78 2020-03-19T23:12:59.525 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : liboath-2.6.2-3.el8.x86_64 72/78 2020-03-19T23:12:59.659 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : librabbitmq-0.9.0-1.el8.x86_64 73/78 2020-03-19T23:12:59.659 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : librdkafka-0.11.4-1.el8.x86_64 74/78 2020-03-19T23:12:59.884 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 74/78 2020-03-19T23:12:59.884 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 75/78 2020-03-19T23:13:00.051 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 75/78 2020-03-19T23:13:00.252 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libxslt-1.1.32-3.el8.x86_64 76/78 2020-03-19T23:13:00.434 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-markupsafe-0.23-19.el8.x86_64 77/78 2020-03-19T23:13:00.594 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : python3-bcrypt-3.1.6-2.el8.1.x86_64 78/78 2020-03-19T23:13:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: python3-bcrypt-3.1.6-2.el8.1.x86_64 78/78 2020-03-19T23:13:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:15.1.1-234.g787db24.el8.x86_64 1/78 2020-03-19T23:13:00.838 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2/78 2020-03-19T23:13:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8. 3/78 2020-03-19T23:13:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 4/78 2020-03-19T23:13:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 5/78 2020-03-19T23:13:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarc 6/78 2020-03-19T23:13:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:15.1.1-234.g787db2 7/78 2020-03-19T23:13:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.no 8/78 2020-03-19T23:13:00.839 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 9/78 2020-03-19T23:13:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 10/78 2020-03-19T23:13:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 11/78 2020-03-19T23:13:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : leveldb-1.21-2.el8.x86_64 12/78 2020-03-19T23:13:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-8.3.1-4.5.el8.x86_64 13/78 2020-03-19T23:13:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 14/78 2020-03-19T23:13:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-8.3.1-4.5.el8.x86_64 15/78 2020-03-19T23:13:00.840 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 16/78 2020-03-19T23:13:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 17/78 2020-03-19T23:13:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 18/78 2020-03-19T23:13:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:15.1.1-234.g787db24.el8.x86_64 19/78 2020-03-19T23:13:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/78 2020-03-19T23:13:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libxslt-1.1.32-3.el8.x86_64 21/78 2020-03-19T23:13:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.3-2.el8.x86_64 22/78 2020-03-19T23:13:00.841 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-threads-0.3.3-2.el8.x86_64 23/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 24/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 25/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 26/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 27/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x8 28/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_ 29/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 30/78 2020-03-19T23:13:00.842 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 31/78 2020-03-19T23:13:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 32/78 2020-03-19T23:13:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 33/78 2020-03-19T23:13:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 34/78 2020-03-19T23:13:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-execnet-1.7.1-1.el8.2.noarch 35/78 2020-03-19T23:13:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 36/78 2020-03-19T23:13:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 37/78 2020-03-19T23:13:00.843 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 38/78 2020-03-19T23:13:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 39/78 2020-03-19T23:13:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 40/78 2020-03-19T23:13:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 41/78 2020-03-19T23:13:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 42/78 2020-03-19T23:13:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-10.0.1-2.el8.noarch 43/78 2020-03-19T23:13:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 44/78 2020-03-19T23:13:00.844 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 45/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 46/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 47/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 48/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 49/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 50/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 51/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 52/78 2020-03-19T23:13:00.845 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-2.6-1.el8.noarch 53/78 2020-03-19T23:13:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 54/78 2020-03-19T23:13:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 55/78 2020-03-19T23:13:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 56/78 2020-03-19T23:13:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 57/78 2020-03-19T23:13:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:15.1.1-234.g787db24.el8.x86_64 58/78 2020-03-19T23:13:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 59/78 2020-03-19T23:13:00.846 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 60/78 2020-03-19T23:13:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 61/78 2020-03-19T23:13:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 62/78 2020-03-19T23:13:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 63/78 2020-03-19T23:13:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 64/78 2020-03-19T23:13:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-3.4.2-12.el8.noarch 65/78 2020-03-19T23:13:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d 66/78 2020-03-19T23:13:00.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 67/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 68/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 69/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 70/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 71/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 72/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 73/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 74/78 2020-03-19T23:13:00.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 75/78 2020-03-19T23:13:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 76/78 2020-03-19T23:13:00.849 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 77/78 2020-03-19T23:13:04.472 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 78/78 2020-03-19T23:13:04.473 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:13:04.688 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:04.689 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:13:04.689 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.689 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:13:04.689 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.689 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.689 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:13:04.689 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.690 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:13:04.690 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:13:04.690 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:13:04.690 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:15.1.1-234.g787db24.el8.noarch 2020-03-19T23:13:04.690 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.690 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-03-19T23:13:04.690 INFO:teuthology.orchestra.run.smithi038.stdout: leveldb-1.21-2.el8.x86_64 2020-03-19T23:13:04.691 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-8.3.1-4.5.el8.x86_64 2020-03-19T23:13:04.691 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.2-3.el8.x86_64 2020-03-19T23:13:04.691 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-8.3.1-4.5.el8.x86_64 2020-03-19T23:13:04.691 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-03-19T23:13:04.691 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.691 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-03-19T23:13:04.691 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.3.1-3.el8.x86_64 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: libxslt-1.1.32-3.el8.x86_64 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.3-2.el8.x86_64 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads-0.3.3-2.el8.x86_64 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: python3-apipkg-1.5-6.el8.noarch 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-03-19T23:13:04.692 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-03-19T23:13:04.693 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.693 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.693 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.693 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-03-19T23:13:04.693 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-03-19T23:13:04.693 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-03-19T23:13:04.693 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-execnet-1.7.1-1.el8.2.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-6.2-6.el8.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-03-19T23:13:04.694 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-03-19T23:13:04.695 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-10.0.1-2.el8.noarch 2020-03-19T23:13:04.695 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-03-19T23:13:04.695 INFO:teuthology.orchestra.run.smithi038.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-03-19T23:13:04.695 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.0.6-13.el8.noarch 2020-03-19T23:13:04.695 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-03-19T23:13:04.695 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-03-19T23:13:04.695 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-03-19T23:13:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-03-19T23:13:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-03-19T23:13:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-03-19T23:13:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-2.6-1.el8.noarch 2020-03-19T23:13:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-03-19T23:13:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-03-19T23:13:04.696 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-03-19T23:13:04.697 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2017.2-9.el8.noarch 2020-03-19T23:13:04.697 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.697 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.697 INFO:teuthology.orchestra.run.smithi038.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-03-19T23:13:04.697 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-03-19T23:13:04.697 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-03-19T23:13:04.697 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.4.1-12.el8.noarch 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-3.4.2-12.el8.noarch 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-03-19T23:13:04.698 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-03-19T23:13:04.699 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-03-19T23:13:04.699 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-03-19T23:13:04.699 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-03-19T23:13:04.699 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-03-19T23:13:04.699 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-03-19T23:13:04.699 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-03-19T23:13:04.699 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:04.700 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:05.224 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:05.799 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:05.800 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:05.800 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:05.800 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:05.834 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:05.835 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:05.835 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2020-03-19T23:13:05.835 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:05.835 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:13:05.836 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm x86_64 2:15.1.1-234.g787db24.el8 @ceph 148 k 2020-03-19T23:13:05.836 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:05.836 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:13:05.836 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:05.836 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 1 Package 2020-03-19T23:13:05.836 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:05.836 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 148 k 2020-03-19T23:13:05.837 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:13:05.838 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:13:05.839 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:13:05.840 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:13:05.840 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:13:05.897 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:13:05.898 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:13:06.778 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:13:06.779 INFO:teuthology.orchestra.run.smithi038.stdout:userdel: cephadm mail spool (/var/spool/mail/cephadm) not found 2020-03-19T23:13:06.779 INFO:teuthology.orchestra.run.smithi038.stdout:userdel: cephadm home directory (/var/lib/cephadm) not found 2020-03-19T23:13:06.779 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:09.180 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:13:09.181 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:13:09.368 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:09.368 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:13:09.368 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:09.369 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:09.369 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:09.934 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:10.686 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:10.686 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:10.686 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:10.722 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:10.727 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:10.728 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2020-03-19T23:13:10.728 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:10.728 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:13:10.728 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:15.1.1-234.g787db24.el8 @ceph 743 k 2020-03-19T23:13:10.728 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:10.728 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:13:10.728 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:10.729 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 1 Package 2020-03-19T23:13:10.729 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:10.729 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 743 k 2020-03-19T23:13:10.729 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:13:10.729 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:13:10.729 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:13:10.755 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:13:10.755 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:13:10.815 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:13:10.855 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T23:13:10.856 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T23:13:11.303 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T23:13:13.427 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:15.1.1-234.g787db24.el 1/1 2020-03-19T23:13:13.427 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:13:13.609 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:13.610 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:13:13.610 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:13.610 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:13.610 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:14.164 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:14.768 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:14.768 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:14.769 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:14.769 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:14.769 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: ceph-mgr 2020-03-19T23:13:14.770 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:14.774 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:14.776 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:14.776 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:15.270 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:15.929 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:15.929 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:15.929 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:15.929 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:15.930 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: ceph-mgr-dashboard 2020-03-19T23:13:15.930 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:15.935 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:15.936 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:15.936 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:16.405 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:17.045 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:17.045 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:17.045 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:17.046 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:17.046 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: ceph-mgr-diskprediction-local 2020-03-19T23:13:17.047 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:17.051 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:17.052 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:17.052 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:17.522 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:18.186 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:18.186 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:18.187 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:18.187 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:18.187 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: ceph-mgr-rook 2020-03-19T23:13:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:18.192 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:18.193 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:18.194 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:18.664 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:19.340 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:19.340 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:19.340 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:19.340 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:19.341 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: ceph-mgr-cephadm 2020-03-19T23:13:19.341 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:19.346 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:19.347 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:19.348 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:19.826 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:20.456 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:20.456 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:20.457 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:20.457 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:20.492 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:20.494 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:20.494 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2020-03-19T23:13:20.494 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:20.494 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:13:20.494 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:15.1.1-234.g787db24.el8 2020-03-19T23:13:20.494 INFO:teuthology.orchestra.run.smithi038.stdout: @ceph 1.9 M 2020-03-19T23:13:20.495 INFO:teuthology.orchestra.run.smithi038.stdout:Removing unused dependencies: 2020-03-19T23:13:20.495 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.7-12.el8 @rhel-8-for-x86_64-baseos-rpms 206 k 2020-03-19T23:13:20.495 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-common x86_64 3.2.1-12.el8 @rhel-8-for-x86_64-baseos-rpms 4.7 k 2020-03-19T23:13:20.495 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:20.495 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:13:20.495 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:20.495 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 3 Packages 2020-03-19T23:13:20.496 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:20.496 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 2.1 M 2020-03-19T23:13:20.496 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:13:20.498 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:13:20.498 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:13:20.538 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:13:20.538 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:13:20.625 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:13:20.634 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/1 2020-03-19T23:13:20.634 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T23:13:20.881 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T23:13:21.046 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T23:13:21.046 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T23:13:21.046 INFO:teuthology.orchestra.run.smithi038.stdout:warning: /etc/fuse.conf saved as /etc/fuse.conf.rpmsave 2020-03-19T23:13:21.047 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:21.412 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T23:13:21.412 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T23:13:21.412 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2020-03-19T23:13:23.848 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-03-19T23:13:23.848 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:13:24.026 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:24.026 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:13:24.026 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:15.1.1-234.g787db24.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2020-03-19T23:13:24.026 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-03-19T23:13:24.027 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:24.027 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:24.589 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:25.226 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:25.227 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:25.227 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:25.227 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:25.263 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:25.264 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:25.265 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2020-03-19T23:13:25.265 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:25.265 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:13:25.265 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:15.1.1-234.g787db24.el8 @ceph 487 k 2020-03-19T23:13:25.265 INFO:teuthology.orchestra.run.smithi038.stdout:Removing dependent packages: 2020-03-19T23:13:25.265 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:15.1.1-234.g787db24.el8 @ceph 76 k 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout:Removing unused dependencies: 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:15.1.1-234.g787db24.el8 @ceph 1.6 M 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 3 Packages 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:25.266 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 2.2 M 2020-03-19T23:13:25.267 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:13:25.268 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:13:25.269 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:13:25.303 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:13:25.303 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:13:25.378 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:13:25.487 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T23:13:25.488 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2/3 2020-03-19T23:13:25.704 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2/3 2020-03-19T23:13:25.815 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T23:13:26.025 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T23:13:26.025 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 1/3 2020-03-19T23:13:26.025 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2/3 2020-03-19T23:13:28.208 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:15.1.1-234.g787db24.el8.x86_64 3/3 2020-03-19T23:13:28.208 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:13:28.416 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:28.416 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:13:28.416 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:28.416 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:28.417 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:28.417 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:28.417 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:28.968 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:29.596 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:29.596 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:29.596 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:29.596 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:29.597 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: libcephfs2 2020-03-19T23:13:29.597 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:29.602 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:29.603 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:29.603 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:30.070 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:30.703 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:30.703 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:30.703 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:30.703 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:30.704 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: libcephfs-devel 2020-03-19T23:13:30.704 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:30.709 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:30.710 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:30.710 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:31.206 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:31.772 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:31.772 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:31.772 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:31.772 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:31.811 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:31.813 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: Package ArchVersion Repository Size 2020-03-19T23:13:31.813 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:31.813 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2020-03-19T23:13:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_642:15.1.1-234.g787db24.el8 @ceph 16 M 2020-03-19T23:13:31.813 INFO:teuthology.orchestra.run.smithi038.stdout:Removing dependent packages: 2020-03-19T23:13:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: qemu-kvm-block-rbd x86_6415:2.12.0-88.module+el8.1.0+5149+3ff2765e.2 @rhel-8-for-x86_64-appstream-rpms 34 k 2020-03-19T23:13:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_642:15.1.1-234.g787db24.el8 @ceph 230 k 2020-03-19T23:13:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_642:15.1.1-234.g787db24.el8 @ceph 526 k 2020-03-19T23:13:31.814 INFO:teuthology.orchestra.run.smithi038.stdout:Removing unused dependencies: 2020-03-19T23:13:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_642:15.1.1-234.g787db24.el8 @ceph 8.6 M 2020-03-19T23:13:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_642.8.1-11.el8 @lab-extras 1.0 M 2020-03-19T23:13:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:31.815 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2020-03-19T23:13:31.815 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2020-03-19T23:13:31.815 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 6 Packages 2020-03-19T23:13:31.815 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:31.815 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 26 M 2020-03-19T23:13:31.815 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2020-03-19T23:13:31.818 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2020-03-19T23:13:31.818 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2020-03-19T23:13:31.854 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2020-03-19T23:13:31.855 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2020-03-19T23:13:31.974 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2020-03-19T23:13:32.127 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 1/6 2020-03-19T23:13:32.261 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 2/6 2020-03-19T23:13:32.420 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+ 3/6 2020-03-19T23:13:32.421 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : librbd1-2:15.1.1-234.g787db24.el8.x86_64 4/6 2020-03-19T23:13:32.646 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:15.1.1-234.g787db24.el8.x86_64 4/6 2020-03-19T23:13:32.647 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : librados2-2:15.1.1-234.g787db24.el8.x86_64 5/6 2020-03-19T23:13:32.647 INFO:teuthology.orchestra.run.smithi038.stdout:warning: file /etc/ceph: remove failed: No such file or directory 2020-03-19T23:13:32.647 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:32.848 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:15.1.1-234.g787db24.el8.x86_64 5/6 2020-03-19T23:13:32.848 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : lttng-ust-2.8.1-11.el8.x86_64 6/6 2020-03-19T23:13:33.327 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/6 2020-03-19T23:13:33.328 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:15.1.1-234.g787db24.el8.x86_64 1/6 2020-03-19T23:13:33.328 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:15.1.1-234.g787db24.el8.x86_64 2/6 2020-03-19T23:13:33.328 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 3/6 2020-03-19T23:13:33.328 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+ 4/6 2020-03-19T23:13:33.328 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 5/6 2020-03-19T23:13:35.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 6/6 2020-03-19T23:13:35.491 INFO:teuthology.orchestra.run.smithi038.stdout:Installed products updated. 2020-03-19T23:13:35.691 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:35.692 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2020-03-19T23:13:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: qemu-kvm-block-rbd-15:2.12.0-88.module+el8.1.0+5149+3ff2765e.2.x86_64 2020-03-19T23:13:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:35.692 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:15.1.1-234.g787db24.el8.x86_64 2020-03-19T23:13:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-03-19T23:13:35.693 INFO:teuthology.orchestra.run.smithi038.stdout: 2020-03-19T23:13:35.693 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:36.247 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:36.817 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:36.817 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:36.817 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:36.817 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:36.818 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: librbd1 2020-03-19T23:13:36.818 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:36.823 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:36.824 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:36.824 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:37.317 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:37.938 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:37.939 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:37.939 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:37.939 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:37.939 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: python3-rados 2020-03-19T23:13:37.940 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:37.944 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:37.946 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:37.946 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:38.445 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:39.060 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:39.060 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:39.061 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:39.061 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:39.061 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: python3-rgw 2020-03-19T23:13:39.062 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:39.066 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:39.067 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:39.068 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:39.562 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:40.153 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:40.153 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:40.153 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:40.153 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:40.154 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: python3-cephfs 2020-03-19T23:13:40.154 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:40.159 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:40.160 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:40.160 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:40.629 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:41.241 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:41.241 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:41.241 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:41.241 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:41.242 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: python3-rbd 2020-03-19T23:13:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:41.247 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:41.248 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:41.248 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:41.754 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:42.291 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:42.291 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:42.291 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:42.291 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:42.292 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: rbd-fuse 2020-03-19T23:13:42.292 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:42.297 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:42.298 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:42.298 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:42.781 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:43.351 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:43.351 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:43.351 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:43.352 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:43.352 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: rbd-mirror 2020-03-19T23:13:43.353 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:43.357 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:43.359 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:43.359 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:43.848 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:44.387 INFO:teuthology.orchestra.run.smithi038.stderr:Modular dependency problem: 2020-03-19T23:13:44.387 INFO:teuthology.orchestra.run.smithi038.stderr: 2020-03-19T23:13:44.387 INFO:teuthology.orchestra.run.smithi038.stderr: Problem: conflicting requests 2020-03-19T23:13:44.388 INFO:teuthology.orchestra.run.smithi038.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-03-19T23:13:44.388 INFO:teuthology.orchestra.run.smithi038.stdout:No match for argument: rbd-nbd 2020-03-19T23:13:44.389 INFO:teuthology.orchestra.run.smithi038.stderr:No packages marked for removal. 2020-03-19T23:13:44.393 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2020-03-19T23:13:44.394 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2020-03-19T23:13:44.394 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2020-03-19T23:13:44.601 INFO:teuthology.orchestra.run.smithi038:> sudo yum clean all 2020-03-19T23:13:44.952 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:45.495 INFO:teuthology.orchestra.run.smithi038.stdout:52 files removed 2020-03-19T23:13:45.539 INFO:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2020-03-19T23:13:45.578 INFO:teuthology.orchestra.run.smithi038.stdout:RedHatEnterprise 2020-03-19T23:13:45.579 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T23:13:45.580 INFO:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/yum.repos.d/ceph.repo 2020-03-19T23:13:45.650 INFO:teuthology.orchestra.run.smithi038:> sudo yum clean expire-cache 2020-03-19T23:13:46.040 INFO:teuthology.orchestra.run.smithi038.stdout:Updating Subscription Management repositories. 2020-03-19T23:13:46.564 INFO:teuthology.orchestra.run.smithi038.stdout:Cache was expired 2020-03-19T23:13:46.564 INFO:teuthology.orchestra.run.smithi038.stdout:0 files removed 2020-03-19T23:13:46.607 DEBUG:teuthology.parallel:result is None 2020-03-19T23:13:46.608 INFO:teuthology.task.install:Removing ceph sources lists 2020-03-19T23:13:46.608 INFO:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2020-03-19T23:13:46.647 INFO:teuthology.orchestra.run.smithi038.stdout:RedHatEnterprise 2020-03-19T23:13:46.648 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-03-19T23:13:46.649 INFO:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/yum.repos.d/ceph.repo 2020-03-19T23:13:46.719 INFO:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/yum/pluginconf.d/priorities.conf.orig /etc/yum/pluginconf.d/priorities.conf 2020-03-19T23:13:46.789 DEBUG:teuthology.parallel:result is None 2020-03-19T23:13:46.790 DEBUG:teuthology.run_tasks:Unwinding manager clock 2020-03-19T23:13:46.807 INFO:teuthology.task.clock:Checking final clock skew... 2020-03-19T23:13:46.807 INFO:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-03-19T23:13:46.843 INFO:teuthology.orchestra.run.smithi038.stderr:bash: ntpq: command not found 2020-03-19T23:13:46.846 INFO:teuthology.orchestra.run.smithi038.stdout:210 Number of sources = 4 2020-03-19T23:13:46.847 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-03-19T23:13:46.847 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2020-03-19T23:13:46.847 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T23:13:46.847 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T23:13:46.847 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2020-03-19T23:13:46.847 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv04.front.sepia.ceph.com 3 6 377 55 +2035ns[+3712ns] +/- 39ms 2020-03-19T23:13:46.848 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2020-03-19T23:13:46.863 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2020-03-19T23:13:46.863 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2020-03-19T23:13:46.876 INFO:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2020-03-19T23:13:46.948 INFO:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-03-19T23:13:46.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T23:13:46.980 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2020-03-19T23:13:46.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2020-03-19T23:13:47.007 INFO:teuthology.task.internal:Duration was 593.347989 seconds 2020-03-19T23:13:47.008 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2020-03-19T23:13:47.021 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2020-03-19T23:13:47.021 INFO:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2020-03-19T23:13:47.079 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-03-19T23:13:47.394 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2020-03-19T23:13:47.394 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2020-03-19T23:13:47.395 INFO:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/*.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2020-03-19T23:13:47.414 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2020-03-19T23:13:47.415 INFO:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2020-03-19T23:13:47.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2020-03-19T23:13:47.510 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2020-03-19T23:13:47.511 INFO:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2020-03-19T23:13:47.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2020-03-19T23:13:47.583 INFO:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2020-03-19T23:13:47.635 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2020-03-19T23:13:47.637 INFO:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2020-03-19T23:13:47.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-03-19T23:13:47.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2020-03-19T23:13:47.704 INFO:teuthology.task.internal:Transferring archived files... 2020-03-19T23:13:47.705 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sage-2020-03-19_17:25:48-rados-wip-sage-testing-2020-03-19-0855-distro-basic-smithi/4868731/remote/smithi038 2020-03-19T23:13:47.705 INFO:teuthology.orchestra.run.smithi038:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2020-03-19T23:13:47.773 INFO:teuthology.task.internal:Removing archive directory... 2020-03-19T23:13:47.773 INFO:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2020-03-19T23:13:47.823 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2020-03-19T23:13:47.844 INFO:teuthology.task.internal:Not uploading archives. 2020-03-19T23:13:47.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2020-03-19T23:13:47.861 INFO:teuthology.task.internal:Tidying up after the test... 2020-03-19T23:13:47.861 INFO:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2020-03-20T10:57:07.967 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-03-20T10:57:07.987 DEBUG:teuthology.task.console_log:Killing console logger for smithi038 2020-03-20T10:57:07.988 DEBUG:teuthology.task.console_log:Killing console logger for smithi038 2020-03-20T10:57:07.988 DEBUG:teuthology.exit:Finished running handlers